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“The Foremost European Electronic Imaging Events in the Visual Arts” 


Forum for Users, Suppliers & Researchers 


The key aim of this Event is to provide a forum for the user, supplier and scientific research 
communities to meet and exchange experiences, ideas and plans in the wide area of Culture & 
Technology. Participants receive up to date news on new EC and international arts computing & 
telecommunications initiatives as well as on Projects in the Visual Arts field, in archaeology and 
history. Working Groups and new Projects are promoted. Scientific and technical demonstrations are 
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Culture and e-government 

Activities and Programmes for e-learning 
Digital TV and films 

3D Developments and Applications in the Cultural Heritage Area 
Virtual Galleries and Exhibitions 
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Digital Theatre 

Cultural Tourism & Travel Applications 
Impact of Culture in the Smart City 
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WHO SHOULD ATTEND 


THE CULTURAL SECTOR: The Visual Arts Community including Museums, Libraries, 
Archaeological Sites, Educational Institutions, Commercial Galleries and Dealers, Auction Houses, 
Artists & Collectors 

THE HI-TECH INDUSTRY SECTOR: Multimedia Systems, Image Acquisition & Analysis, Data- 
bases, Display & Printing, ICT Industry, Telematics & Systems Manufacturing, On-line Information 
Services 

MEDIA & RELATED SECTORS: Publishing, Press, Film, Television, Photography, Printing, 
Advertising, Graphics Design, Consumer Media 

IMAGING SYSTEMS RESEARCHERS: Imaging Systems, 3-D Acquisition, Reconstruction & 
Representation Systems, Information Sciences 

TOURISM & TRAVEL SECTOR: Tourism Agencies & Operators, Travel Agencies 

THE GOVERNMENT SECTOR: Ministries of Culture and other Institutions involved in Cultural 
Heritage, Ministries of Industry, Education, Research and Science, Regional Governments 
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PROGRAM - PLANNING 


Wednesday, 9 May 


9,30 — 13,00 WORKSHOP p. 11 
14,15 — 15,30 OPENING OF CONFERENCE p. 12 
15,45 - 17,15 SESSION 1 p. 12 
17,15 — 18,45 SESSION 2 p. 13 
11,00 — 19,00 TECHNICAL EXHIBITION p. 19 
19,00 — 22,00 SPECIAL EVENT p. 19 


Thursday, 10 May 


9,30- 11,45 INTERNATIONAL FORUM p.15 
ON "CULTURE & 
TECHNOLOGY" 
12,00 — 13,15 SESSION 3 p. 16 
14,40 — 16,40 SESSION 4 p. 16 
16,55 — 18,40 SESSION 5 p. 17 
10,00 — 18,00 ART EXHIBITION p. 19 
Venue: Hotel Pierre 


Via De’ Lamberti, 5 - 50123 Firenze 

Tel. +39 055 216218 Fax +39 055 2396573 
E-mail: pierre@remarhotel.com 
http://www.hotel-pierre-florence.com/ 
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WORKSHOP 


ROOM A 

WORKSHOP INNOVATION AND ENTERPRISE - INNOVAZIONE E IMPRESA 
(Italian Language) 

9,30 — 13,00 

Chairman: Enrico Bocci, Vice-Presidente Confindustria Firenze, Firenze 


Technological requirements in the Cultural Heritage field are outlined and opportunities for 
Italian Enterprises and SME’s working in the field, using new technologies, are presented. 
Regional and National Applied Research Programs in Italy are described. 

Activities by National Organizations and Firms working in the area of Telecommunications, 
Informatics, Environment and Infomobility are presented. 

Funding by European Commission is considered, with particular reference to multimedia 
and telematics for Cultural Heritage. Special consideration is given to the EC Plan 
HORIZON 2020. 

Initiatives regarding the “know-how” transfer from Research Organizations to the Industrial 
Sector are described, in particular to create Start-Ups and new Enterprises. 


Organizations and Companies present their activities and experiences. 


Invited Speakers: - Andrea Arnone, Pro-Rettore al Trasferimento Tecnologico e 
Presidente di CsaVRI, Università degli Studi di Firenze 
- Laura Castellani, Responsabile del Settore Infrastrutture e 
Tecnologie per lo Sviluppo della Società dell’ Informazione, 
Regione Toscana 
- Paola Castellacci, VAR GROUP 


Speakers include:  - Stefano Cinquini, TELECOM ITALIA 
- Renzo Zampini, INFOCAMERE 
- Andrea Calistri, SAPAF Srl, Firenze 
- Riccardo Bruschi and Luca Bencini, T. T. TECNOSISTEMI, Prato 
- Andrea Sbandati, CISPEL, Confservizi TOSCANA, Firenze 
- Gianluca Vannuccini, Servizio Sviluppo Infrastrutture Tecnologiche, 
Comune di Firenze, Firenze 
- Francesco Mati, Piante MATI, Pistoia 
- Claudio Tasselli, Qu.In. Srl, Calenzano, Firenze 
- Franco Guidi, NEUMUS, Firenze 
- Massimo Canalicchio, Mentor Incubatore di Firenze, Firenze 
- Marco Cappellini, CENTRICA Srl, Firenze 


Closing: - Cecilia Del Re, Assessore Sviluppo Economico, 
Comune di Firenze, Firenze 


13,00 Lunch Break 
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ROOM A 


CONFERENCE 


Wednesday, 9 May 


Chairmen: Vito Cappellini, University of Florence 
Enrico Del Re, University of Florence 


14,15 Opening: James Hemsley, 
EVA Conferences International 
Enrico Vicario, 
Director of Department of Information Engineering — 
University of Florence 
Gabriele Gori, 
General Director Fondazione Cassa di Risparmio di Firenze 
Paolo Castellacci, 
President GRUPPO SESA 


15,30 Coffee Break 
ROOM A 
15,45 SESSION 1 - STRATEGIC ISSUES 


Chairman: Paolo Blasi, University of Florence, Florence, Italy 


Bahar Taskesen!, 
A G Constantinides* 
*Middle East Technical University, Turkey 


“Interactive Image Reconstruction from its 
Incomplete, Irregular and Imprecise 


Fragments” A 
Imperial College London, U.K. 

“At the Intersection of Art, Architecture Herbert D. G. Maschner!, 

and Archaeology: 3D Virtualization and Victor Manuel López-Menchero Bendicho!, 

Contemporary Heritage" Miguel Ángel Hervás Herrera?, Jeffrey Du 
Vernay!, Aurelia Lureau*?, James Bart McLeod! 
! Global Digital Heritage, St Petersburg, 
Florida, U.S.A. 
? Baraka Arqueólogos SL, Ciudad Real, Spain 
Université Paris — Panthéon-Sorbonne, France 

“Is three better than two? Claudio Lucchiari, 

A study on EEG activity and imagination Maria Elide Vanutelli, 

abilities in 2D vs 3D stimuli” Raffaella Folgieri 


Department of Philosophy, 
Universita degli Studi di Milano, 
Milan, Italy 
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“A Scientific Method for the Attribution of 
Paintings with application to Leonardo’s 
Mona Lisa twins” 


“Use of Multimedia and Virtual Reality 
Technologies to Represent Russian 
Cultural Heritage” 


ROOM A 


17,15 
APPLICATIONS 
Chairman: 


“Homo Technologicus comes of Age: 
An Ongoing Evolution” 


“Cognitive stages in rational thinking - toward 
human technology” 


“A new Compact VNIR Hyperspectral Imaging 
System for Non-Invasive Analysis in the 
FineArt and Architecture Fields” 


J.F. Asmus!, 

Vadim Parfenov? 

‘Department of Physics & Center for Advanced 
Nanotechnology, University of California, CA, U.S.A. 
¿Department of Quantum Electronics and Opto- 
Electronic Devices, Saint Petersburg 
Electrotechnical University 

Saint Petersburg, Russia 


Nikolay Borisov*”, 

Artem Smolin”, 

Vera Slobodyanuk®? 

Dept. of Information Systems for Arts and 
Humanities, Saint Petersburg State University 
*Dept. of Graphic Technologies, Center for Design 
and Multimedia, ITMO University 

Saint Petersburg, Russia 


SESSION 2 - NEW SCIENCE AND CULTURE DEVELOPMENTS & 


Edoardo Calia, Istituto Superiore Mario Boella (ISMB), Torino, Italy 


Giuseppe O. Longo 
University of Trieste 
Trieste, Italy 


Mika Laakkonen 
Faculty of Social Science, 
University of Lapland 
Lapland, Finland 


M. Picollo’, A. Casini’, C. Cucci’, J. Jussila?, 
M. Poggesi!, L. Stefani! 
t Nello Carrara Institute for Applied Physics of the 


Italian National Research Council (IFAC-CNR) Florence, 


Italy 
? SPECIM, Spectral Imaging Ltd. 
Oulu, Finland 
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“The place of inspiration of the Flemish 
Triptych by Rogier van der Weyden. A 
contribution of the landscape busting 
to one of the vexata quaestio of the 
Sicilian history” 


“The Cultural Heritage of tomorrow: should 
we put a limit to the influence that new 
technologies have on culture and design?” 


“The use of the intelligent cutting guide 
PERSEUS during Total Knee Replacement” 


R. Franco 
Societa Italiana di Geologia Ambientale 
Gangi, Italy 


Gianpiero Alfarano, Erika Lascialfari 
DESIGN CAMPUS, 

University of Florence 

Calenzano - Florence, Italy 


Lawrence Camarda!, 

Antonio D'Arienzo!, 

Michele D’Arienzo* 

*Orthopaedic and Traumatology Department, 
University of Palermo 

Palermo, Italy 
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ROOM A 


9,30 


The structure of the FORUM is presented. 


Thursday, 10 May 


INTERNATIONAL FORUM ON “CULTURE & TECHNOLOGY 
Chairman: Vito Cappellini, University of Florence, Florence, Italy 


Actual developments and perspectives are outlined, regarding Culture and Technology. 


Opening: 


Speakers Include: 


11,45 


Coffee Break 


Cooperation Groups 
Proposed Projects 

Funding Opportunities 
European Commission Plans 


- Eugenio Giani, President Consiglio 
Regionale, Regione Toscana, Italy 


- Cristina Acidini, President Accademia delle 
Arti del Disegno, Florence, Italy 

- Edoardo Calia, Research Director, Istituto 
Superiore Mario Boella, Turin, Italy 

- Alberto Del Bimbo, Director Centro per la 
Comunicazione e l'Integrazione dei Media, 
University of Florence, Florence, Italy 

- Veronica Elena Bocci, 

Coordinator DITECFER, Pistoia, Italy 

- David Feldman, Vice President, 

THE MONA LISA FOUNDATION, Zurich, 
Switzerland 

- Monica Carfagni, Full Professor of Industrial 
Engineering, University of Florence, Florence, 
Italy 

- Carlo Francini, Florence Municipality, 
Florence, Italy 

- Paolo Zampini, Director of Conservatorio di 
Musica Luigi Cherubini, Florence, Italy 

- Emiliano Degl'Innocenti, DARIAH-IT National 
Coordinator, Consiglio Nazionale delle 
Ricerche, Italy 

- Francesco Bellini, University of Rome "La 
Sapienza" and Research Director of 
EUROKLEIS, Rome, Italy 

- Francesca Gemma, Aracne editrice int.le, 
Rome, Italy 

- Giovanni Gasbarrone, CIU Confederazione 
Unione delle Professioni Intellettuali, 
Responsabile Lazio, Italy 
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12,00 
Chairman: 


“Latest Innovation on Capturing Historical 
Photographic Ethereogenous Material: 
The Alinari Experience” 


“International Standardization of FTV” 


"4D Ray-Space and Ultra-Wide Area FTV” 


“Digital innovation & Technological explosion 
The New Challenges of the Security 
Management” 


“New Trends of 3D Technologies and 
Copyright Protection” 


SESSION 3 - NEW TECHNICAL DEVELOPMENTS & APPLICATIONS 
Andrea De Polo Saibanti, Fratelli Alinari IDEA Spa, Florence, Italy 


Andrea De Polo Saibanti 
Fratelli Alinari IDEA SpA 
Florence, Italy 


Masayuki Tanimoto 

Nagoya University and Nagoya Industrial Science 
Research Institute 

Nagoya, Japan 


Masayuki Tanimoto’, Hirokuni Kurokawa? 
‘Nagoya Industrial Science Research Institute, 
Nagoya, Japan 

"University of Aizu 

Aizu Wakamatsu, Japan 


Franco Guidi, Giancarlo Caroti 
Neumus SrL 
Florence, Italy 


Vito Cappellini’, Francesca Uccheddu? 
University of Florence and INN-3D 

Italy 

?INN-3D and Department of Industrial Engineering- 
University of Florence and INN-3D Srl 

Italy 


13,15 Lunch Break 

ROOM A 

14,40 SESSION 4 - CULTURAL ACTIVITIES — REAL AND VIRTUAL 
GALLERIES AND RELATED INITIATIVES 

Chairman: 


Paris, France 


"Museo Nazionale del Bargello" 


Jeanette Zwingenberger, Université Paris 1 Panthéon-Sorbonne, 


llaria Ciseri 
Museo Nazionale del Bargello 
Florence, Italy 
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“Resisting a Total Loss of Digital Heritage 
Web 2.0 - Archiving & Bridging Thesaurus 
for Media Art Histories” 


“The extraordinary role of the imaging 
techniques in the conservation and 
valorization of Cultural Heritage” 


“Technology and theatrical tradition in art 
exhibitions: "Earlier Mona Lisa" in Shanghai” 


“Renaissance Experience: Florence and Uffizi” 


“Smarticon: A Digital Eco-System for Cultural 
Heritage. Iconographic Convergences in Art 
and in World Religious” 


“Some Reflections & Questions on Emerging 
Forms of Digital Photo-Libraries” 


“Contemporary Art Creation 
and Exhibitions” 


16,40 Coffee Break 


ROOM A 


16,55 
Chairman: 


“Creative perception, decoding of hidden 
images, contextualisation and interactive 
learning process” 


“Instant Architecture: Hosted Access to the 
Archivision Research Library 

with Built-In Image Management & 
Presentation Tools” 


Oliver Grau 

Department for Image Science, 
Danube University 

Krems, Austria 


Mauro Matteini 
ICVBC - C.N.R. 
Florence, Italy 


Jean Paul Carradori, Qing Li 
Shanghai International Interior Design Festival 
Europe area 


Marco Cappellini, Paolo De Rocco, 
Paolo Romoli 

Centrica Srl, Murate Idea Park 
Florence, Italy 


Sara Penco 
Start up "Sapere Project" 
Rome, Italy 


James R. Hemsley 
Birkbeck College 
London, U.K. 


Paola Imposimato 

Studio Creazioni di Design e Arti Grafiche e 
Pittoriche, 

Florence, Italy 


SESSION 5 - ACCESS TO THE CULTURE INFORMATION 
James Hemsley, EVA Conferences International, U.K . 


Jeanette Zwingenberger 
Université Paris 1 Panthéon-Sorbonne 
Paris, France 


Maureen Burns!, Andreas Knab? 
‘Archivision Associate & IMAGinED Consulting 
Los Angeles, U.S.A. 

^Owner vrcHost LLC 

Miami, Florida, U.S.A. 
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“Maximizing Metadata: Embedded 
Metadata Tools” 


"Il Piccolo Masaccio e le Terre Nuove. 
Creativity and Computer Graphics for 
Museum Edutainment" 


“Canals, Cities, Museums, Libraries & 
Photography: A Reconnaissance Study of 
Regent’s Canal, London” 


“3D Visualization: Revealing Imagery Space by 
Technological Parameter” 


“A Techno Social Collaborative Platform 
to Manage Optimize and Crowdfund 
Cultural Heritage Initiatives” 
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Marcia Focht 

Department of Art History, 
Binghamton University 
Binghamton, New York, U.S.A 


Valentina Zucchi’, Antonella Guidazzoli?, 
Giovanni Bellavia?, Daniele De Luca?, 

Maria Chiara Liguori?, Francesca Delli Ponti?, 
Federica Farroni?, Beatrice Chiavarini? 
*Museo Terre Nuove 

San Giovanni Valdarno, Italy 

*Cineca 

Bologna, Italy 


Graham Diprose!, Christina Hemsley?, 
James R. Hemsley? 

Photographer, Author & Lecturer 
London, U.K 

*Digital Consultant & Photographer 
London, U.K 

*Birkbeck College 

London, U.K. 


Nina Sosna 

Department of Visual Anthropology, 
Institute of Philosophy 

Moscow, Russia 


F. Spadoni!, F. Tariffi?, 
R. Rossi, S. Lusso” 
‘Rigel Engineering S.r.l. 
Livorno, Italy 

“Space S.p.A. 

Prato, Italy 


SPECIAL EVENT 


Wednesday, 9 May: 19,00 - 22,00 


Visit to Art Collection of FONDAZIONE CR FIRENZE 


EXHIBITION 
ROOM B 


Wednesday 9 May: 11,00 - 18,00 


TECHNICAL EXHIBITION 


Augmented Virtual Reality Projects-Realizations and 3D Digital Technologies will 
be presented with Technical Demonstrations. 


Documentation on TECHNICAL EXHIBITION will be available. 


Thursday 10 May: 10,00 — 18,00 


ART EXHIBITION 


Organizer and Coordinator: Riccardo Saldarelli, Presidente della Sezione Artisti 
della ANTICA COMPAGNIA DEL PAIOLO (Presidente Anna Bini) 


Riccardo Saldarelli has given very important Contributions to “Digital Art’, since 
1970. He will present many of his Paintings, regarding in particular “Digital Art” 
Creations, by using large Digital Displays. Also Giusy Celeste and Jannette 
Rutsche will present their “Digital Art” Works. 


Paintings of other Artists will be also shown. 
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At 16,30 Franco Samoggia (Sezione Scienze), introduced by Riccardo Saldarelli, 
will present his Book “IL VIZIO DI LAVORARE”, making also reference to a special 
Medal - created by Riccardo Saldarelli — which has travelled on a “Spacecraft”. 


Documentation on ART EXHIBITION will be available. 


For information on the EXHIBITION: 
CENTRICA S.r.l. 


Piazza della Madonna della Neve, 5 - 50122 Firenze - Italy 
Tel.: (+39) 055 24 66 802 — Fax: (+39) 055 20 09 785 
E-mail: info@evaflorence.it ^ Web pages ~ http://www.evaflorence.it/home.php 


Events Organization: INN-3D Srl, Empoli (FI) 


Official Television TVL Spa Pistoia 
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NEXT EVA EVENT 


9-12 July 2018 EVA London 2018 


Web Site: http://www.eva-london.org 
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PROCEEDINGS 


V. Cappellini (a cura di), Electronic Imaging & the Visual Arts. Eva 2018 ISBN (print) 978-88 6453 706-1, 
ISBN (online) 978-88-6453-707-8, CC BY 4.0, 2018 Firenze University Press. 


STRATEGIC ISSUES 


V. Cappellini (a cura di), Electronic Imaging & the Visual Arts. Eva 2018 ISBN (print) 978-88 6453 706-1, 
ISBN (online) 978-88-6453-707-8, CC BY 4.0, 2018 Firenze University Press. 


Interactive Image Reconstruction from its Incomplete, 
Irregular and Imprecise Fragments 


Bahar Taskesen,' A G Constantinides’ 


' Middle East Technical University, Turkey 
? Imperial College London, U.K. 


1 Introduction 


The aim of this paper is to present a novel method as an adjunct to a 
human/expert for assembling unknown images from their fragments. Reassembling an 
image from its partial and eroded or decayed constituent fragments is of great 
importance in variety of fields, such as forensics, failure analysis, anthropology, 
archaeology and art reconstruction. It is a common task, indeed, in archaeological 
research and artefact preservation. Reassembly from archaeological fragments is a 
much more involved problem in comparison to other fields due to the following 
reasons. The number of randomly mixed fragments is normally huge, the fragments are 
mostly corrupted, irregular, and frequently with uncertain and eroded boundaries. In 
archaeology the process of assembling may not be required to produce a complete 
solution. Indeed, it may be required to produce a series of possible options and even 
partial solutions which the archaeologist would appraise subject to a range of 
archaeological and often unquantifiable constraints. We propose such a method for 
archaeological images. 

This paper is organized as follows. Prior work related with our proposed method is 
shared in Section 2, while detailed explanation of the proposed method is presented in 
Section 3. Finally, this paper ends with the conclusion part in the Section 4. 


2 Related Work 


The fact that reassembly of an unknown image manually from its irregular and 
very often eroded fragments is very time-consuming task, motivates researchers to 
design automated systems [1], [2], [3], [4]. The algorithms produced are usually built 
upon several subsystems which generate reliable or accurate scores between fragments. 
Although, these algorithms show great success in specific cases, they have, so far, 
limited success in reassembling incomplete images with vast numbers of eroded pieces 
automatically. 

For 2D matching the algorithms proposed so far can be divided into two categories: 
a) colour based methods, and b) geometry based approaches. Colour based methods use 
the colour information present in the fragments and mainly at the boundaries [4], [5], 
[6], [7]. These algorithms are mostly efficient, but they fail when only the mean of the 
colour intensities belonging to a fragment is taken, or only when the boundary of the 
pair of fragments is considered. Geometry based approaches assemble the fragments by 
matching the boundary curves of the fragment pairs [8], [7], [9], [10]. The geometry 
based approaches can accurately match the adjacent pairs if there is no corrupted or 
eroded geometry at the broken parts of the fragments. Since the fragments of interest in 
our case have exposed to the natural elements, perhaps for thousands of years, it is 
inevitable that the edges of the pieces are eroded/corrupted, and their geometry is 
thereby destroyed. Moreover, the geometry based approaches are relatively slow as 
compared to other methods and they can fail in their search to lock at a local minimum 
of the objective function. 

The latest research published focussed on initially finding matches between the 
fragments by checking the patterns at the boundaries of adjacent fragments. This 
pattern alignment is enhanced by several methods which can be listed briefly as 
follows, a) by using the colour information [11], b) by describing fractured edges using 
the new ‘ribbon’ idea [12], c) by normal maps [13]. These methods have great success 
when the features of the fragment pairs are significantly different. We construct an 
integrated approach with the above-mentioned methods, in which mostly the boundary 
correlations between the potential pairs dominate the matches. Matching scores are 
derived and the scores between the potential pairs are adjusted with each of the several 
algorithmic iterations in a graph based approach. 
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3 Proposed Method 


Our proposed method is not meant to replace the experts” (in our case the 
archaeologist’s) skills and knowledge but rather to be an aid and an additional tool. 
This aspect is incorporated in the structure of the algorithm. In our approach we support 
the most likely match with periodicity check, line continuity and inner layer correlation 
all of these terms are explained in this paper below. At the end, the most meaningful 
match is selected as decided by the expert/archaeologist. The method is explained in the 
following subsections which cover formulation of the problem, border and boundary 
extraction, score assignment, decision unit and score rearrangement. 


Formulation of the Problem 


Given the set of fragments G = G; of the image (I), we wish to find the most 
likely match between fragments. In the composition of the image I each fragment is 
identified by its pixel boundary vector Bi, i = 1...N, where N is the total number of 
available fragments. The image (1), has more fragments than N, such that some of the 
fragments are lost, however, only N number of fragments are accessible. Because of 
the possibly of imprecise, eroded or corrupted boundaries we also check the inner 
layers. A layer is a sequence of pixel values once or several times removed off the 
boundary of a given fragment. Hence, each fragment is defined by its boundary matrix 
for each layer, Bi; where j is the layer number /.../y, and ly is the maximum layer 
number. To locate the maximum correlation exactly in the whole boundary matrix of 
the fragment we also detect the corners (C) of a fragment and divide the boundary 
vector into borders (sub-boundaries) Fx, where k = 1...C, Fx Bi) Gi UL as shown in 
the Figure 1. 

Another reason for dividing the boundary matrix into borders is the fact that each 
boundary matrix is huge compared to the likely correct matched pairs. This result in 
vast and unrelated parts of the boundary matrix. These unrelated parts decrease the 
correlation between correctly matched pairs and give misleading results subsequently. 
To avoid these problems, we prefer to check the relatively small but reasonable portions 
of the boundary vector/matrix borders as explained in the following subsection. 


Border and Boundary Extraction 


As already mentioned, each fragment is defined by its boundary and related 
borders for the layer of interest. To extract the boundary of the image we transform the 
image into binary (for instance the fragment is all white and the background is black), 
by knowing that the photographic image of each fragment is acquired at a constant 
colour background. The structuring element disk with the distance from its origin (r) 
proportional to the intended layer number is used for morphological erosion operation. 
We simply take the difference of the binary images after two consecutive erosion 
operations, with r set equal to the desired layer number, /y and /y -/. Then we obtain the 
raw boundary vector, without corner indicators. 
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Figure 1: Border Extraction and Boundary Gathering as a Vector Processes Block Diagram 


27 


Commonly used and well-known functions for corner detection work well if the 
fragment is close to rectangular in shape with corners at approximately right angles. 
However, when we have a fragment with other form as in Figure 2a these methods fail 
and produce false corner alarms, such as in Figure 2b. This is essentially due the fact 
that we are working in the discrete domain where there are no continuous lines. As a 
result, when we zoom in the one portion of the fragment as in Figure 2b. Hence for the 
corner detection purposes we designed our own specific algorithm. 


Black crossings are the desired corners and Red crossings are the wrong corners 


a) bi 


Figure 2: a. Fragment Sample with the desired corners b. Closed portion of the 
fragment and the corners detected with the current algorithms 


We obtain a slope plot by using the boundary coordinates as in Figure 3. Corners 
are checked both from periodicity of the slope vector in Figure 3 within the boundary 
vector, and maximum-minimum slopes in a local area of corner. 

In digital world, a line can be defined by its slope period shorter than its size, 
meaning that each line has a periodic behaviour that determines its starting point to end 
point slope. The line cannot follow same the same slope at each consecutive point as it 
can be seen at Figure 2(b), however it follows a period throughout the line. The 
proposed corner detection algorithm is designed based on this claim. If we assume that 
a boundary vector is taken, and its slope between two cascaded points are calculated, 
this brings a slope vector (s) of that boundary as it can be seen at Figure 3. Initially, 
enough data points (m1) depending on the least border size that is investigating, are 
taken from the boundary slope vector. After that, period of this data array (s(1)...s(m1)) 
is calculated, the calculated period of this line is less than its size (m1). Later on, the 
data is accumulated from “s” vector one at a time, and period of the accumulated data 
array (s(1)...s(ml), s(ml+1), s(m1+2)...s(m2)) is calculated. Moreover, if the 
calculated period is still less than the size of the array (m2), then this indicates that 
“s(m2)”, belongs to the line. On the other hand, if the period is equal to the size of the 
array (m2) than this indicates that the line might be following another direction, starting 
from m2. This doubt about m2, that whether it is a corner point, or not, is overcome by 
accumulating the rest of the s vector size at a time, and its corresponding period, as it is 
done until m2 point. Furthermore, if they are equal in every time, then this indicates that 
slope vector is aperiodic after m2, and it is unique. After m2 is marked as a corner, the 
algorithm is iteratively computed for the s’ vector, which corresponds to [s(m1+1), 
s(m1+2)....s(end)]. 

Within the local area next maximum-minimum and the previous maximum- 
minimum slope values are checked and the detected corner is verified. This approach is 
also supported by the local standard deviation calculations of the differences at 
boundary's x values and y values. Peaks in the sliding standard deviation vector over the 
slope vector indicate a possible corner. These two checks, supports the proposed corner 
detection algorithm for the worst cases that it might fail. 

Corner extraction results for an ideal fragment is shown in Figure 4(a), and corners 
for a fragment with curves can be found at Figure 4(b). We put an indicator (NaN in our 
case) for the detected corners in the boundary vector. Hence, we obtain a single vector 
to represent each Gi. 
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Score Assignment 


We used a dynamic graph based approach to track the relations between 
fragment pairs, and their corresponding borders. We basically use a cross correlation 
metric to determine the weights in the dynamic score graph. 

Cross correlation calculated from the equation 1, where Cyiy2 is the sample 
covariance function calculated from 2 and sy), sy; are corresponding sample standard 
deviations of the time series yı and y at lags k = 0, +1, +2, ... calculated from 
equation 3. y, and uy, are the sample means of the time series. 


dj = O k = 0,+1,+2,.. (1) 
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Figure 3: Slope Plot of the Consecutive Points at the Boundary of a Fragment 


Initially, the scores are calculated over the first layer. Elimination with a 
predetermined threshold is performed and most likely solutions are gathered. With this 
knowledge of the most possible matches, we supervise the inner layer correlations. 
However, the correlations at the inner layers are relatively small as it can be seen at the 
Figure 5. As a result, the threshold that eliminates the wrong matches and indicates 
frame borders of the image, (at the frame borders of the overall image I, we do not 
expect any match) is decreased. The whole idea is represented in the Figure 6. 


(a) (b) 


Figure 4: Corner detection for the non-ideally shaped fragment and for the ideally shaped fragment 
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Layer by Layer Score Vectors 
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Figure 6: Block Diagram of the Proposed Method 
Interpolation 


At the inner layers the size of the corresponding border vector is smaller (fewer 
samples). This reduction in the length of the inner layers (pixel changes in the layer 1 to 
layer 3 cross correlation calculation) results in changes of the maximum correlation 
location within the two corresponding borders. To overcome this problem, the smaller 
border vector is linearly extrapolated |layerl - layer2| times. Here, layerl is the layer 
number of the first border and layer2 is the layer number of the second border. 


Calculation of the Cross Correlation in 3D 


Cross correlation is calculated between each fragment pairs at the desired layer 
(j). Especially, in each fragment pair (al and a2), all Fis of the related Ba;j and Ba;j. 
However, since RGB colour space intensities are used throughout the work presented, 
three dimensional vectors are obtained for the boundary vectors. The cross correlations 
between the corresponding channels of the border vectors are calculated. (red channel 
of al - red channel of a2) Then, each cross-correlation vector for red, green and blue for 
the related pair at the predetermined layer are multiplied and to yield a score vector. 
Since, the orientations of the fragments are unknown, the cross correlations should be 
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controlled in two directions. First, the cross correlation is calculated in the same manner 
as the borders located in the boundary vector, and then one of them is reversed and then 
the cross-correlation calculation is performed again. 


Storage of the Data 


Five-dimensional arrays are used to store the scores, and for the graph theory 
applications, implication tables are constructed. The 3D illustration of the array is 
shared in Figure 7. In the score array, 2nd dimension is used to construct the implication 
table. In the 3rd dimension, the cross-correlation multiplication in R-G-B channels 
vector (score vectors) are stored. The 4th dimension is used to store the score vectors 
layer by layer. To be more specific, if ly is the last layer that is going to be used, then 
4th dimension simply constructs the score vectors for the following combinations: 11, 
12, 13... 1ly; 21, 22,...,2ly;ly1,ly2,...lyly. As mentioned earlier, the score vector is 
calculated first with the same locations of the borders vectors as they are in their 
boundary vectors and secondly, with one of the border vector is flipped. Hence, another 
5th dimension is created for the same implication table, scores vector and layers in 
which one of the border vector is reversed. 
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Figure 7: 3D Score Array Illustration 
Periodicity of the Lines 


The periodicity of lines detection is also used as a metric which strengthens the 
score between two fragments. This signals that there is a tight correlation between the 
boundaries of two fragments. The detection is accomplished by checking the scores 
vector (product of cross correlations in R-G-B channels). When there are symmetrical 
peaks around the maximum point, as in Figure 8(a), it can be deduced that precise shifts 
(amount of the period) of the vectors results in again high correlation, which can be 
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seen at the Figure 8(b). Then, after periodicity detection, the score between the related 
fragments is increased exponentially. 


Decision Unit and Score Rearrangement 


The score assignment initially starts automatically. First, the orientation of the 
vectors of interest are decided by checking the maximum values of the scores vector 
calculates between the 1st layers of the fragments. The correct orientation (whether the 
vector is reversed in the boundary vector, or not) although the fragments are not 
matched at all, yields higher acme among the score vector than the uncorrected 
orientation of the two border vectors. Subsequently, we also encode the correct 
orientation of the vectors to the scores array. 

The designed algorithm iteratively goes over the layers up to the predetermined 
ly (maximum number of layers to be used) as it is illustrated in the Figure 8 for the 
most likely matches. The peak value of the score matrix in each layer is checked by 
the threshold and the shape of the score matrix is also considered as a decision metric. 
Mostly for the correct matches, the score vector shows an impulsive behaviour around 
the principally correlated locations of the border vectors. Moreover, at the inner layers 
of the score vector, it is expected that the impulsive behaviour will continue with a 
lower peak, but with the almost same principal location as it can be seen at Figure 5. 
However, for the wrong matches this impulsive behaviour is not observed. 

The algorithm presents the most likely matches to the expert/archaeologist after 
the steps shown in Figure 6, starting from the furthest correlated fragments. 
Knowledge of the context, such as the archaeological excavation environment, 
historical cognisance and human intelligence here take place and decide whether the 
proposed match is correct or not. With this decision the scores are re-adapted and if 
the given match by the algorithm is rejected the corresponding score becomes NaN, 
and not used again. 
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Figure 8: Plot of the score vector (a) and the periodic lines that we detected by our algorithm(b) 
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4 Conclusion and Future Work 


In this paper we presented a novel interactive image reconstruction algorithm. 
Our method consists mainly 3 parts. First, we extract the boundary vector and the 
corners of the fragments. Secondly, we construct a graph based on the correlation 
between the inner boundaries of the fragments. Afterwards, the expert/archaeologist is 
introduced to contribute to the process and to control the proposed algorithmic matches. 
Our results from initial trials show that the correct matches are determined. 

Although our study is only a small step, we believe that this model will insert 
another insight, and will be useful for the construction of future image reassembly 
algorithms. In the future, the 3D space orientation of the fragments can be added as a 
metric for the matches of fractured pairs. Moreover, this method can be applied to the 
complex data-sets with imprecise boundaries and locally corrupted 3D fragments. 
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Abstract — We are at a global transition where disciplines from art to computer engineering 
intersect in the realm of global digital heritage. This has been facilitated by the development of 
desktop high-speed computing, inexpensive photogrammetry software, and digital photography. 
These technologies, and the tools to make them useful both in the lab and on the web, require 
the appropriate integration of technical skill, artistic license, archaeological background 
knowledge, and architectural realities. 


INTRODUCTION 


The documentation of archaeological sites, historic buildings, museums, museum collections, 
works of art, and other cultural heritage resources using 3D acquisition techniques is now 
mainstream practice in most areas of the world. The growing prevalence of on-line resources 
and inexpensive software have allowed students, amateurs, and many professionals the ability 
to collect data and create 3D models of a wide range of subjects, places, and landscapes. 
Photogrammetry in particular, has democratized the participation in 3D model development, 
and nearly every student of digital heritage, every laboratory and university with an interest 
in heritage, and every local, regional, and national government is now participating. With a 
complete photogrammetric kit, including DSLR camera, UAV, computer and software, now 
available for less than $5000, indeed, the basic acquisition of 3D data through 
photogrammetry has been truly democratized. 


But moving beyond basic acquisition and processing into the realm of archaeological 
analyses, architectural reconstruction, and conservation: the science we can create from these 
data, requires a much larger investment. As Dr. Fabio Remondino (FBK) has recently argued 
so cogently, oftentimes the data collected by those not professionally trained, and the 
subsequent 3D models, are not useful for further analyses [1]. Problems of scale, camera 
settings, software settings, computing power, analytical training, and basic photographic 
skills are all important, but the lack of clear documentation of these attributes is equally 
problematic making them less useful for comparative analyses and conservation efforts [2, 3]. 
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For many technical analyses, especially in conservation and architecture, more sophisticated 
data are needed. Terrestrial laser scanners, UAV-based LiDAR and high-resolution 
photogrammetry, and the computing power to process and preserve those data are far beyond 
the possibilities (and finances) of most individuals and labs, and require a significant amount 
of training. But spatial accuracies and geographic controls approaching 1-10 mm are not 
possible using basic photogrammetric kits when we try to digitize large monuments and sites. 
These technologies, and the tools to make them useful both in the lab and on the web, require 


the appropriate integration of technical skill, artistic license, archaeological background 
knowledge, and architectural realities. They also require a rigorous and detail workflow to 
produce results that are verifiable, accurate, and useful for further analyses [4]. 


DISCUSSION 


At Global Digital Heritage, nearly 50 years of combined experience has created a workflow 
that has solved the problems of quality acquisition, data processing, visualization, and data 
access. But this is a dynamic process. Our methods integrate terrestrial and aerial 
photographs, terrestrial and aerial LiDAR, GIS, Google Earth, and other geospatial data and 
imagery. As new algorithms, software, hardware, and ideas are created, these are integrated 
into the workflow to create new opportunities for digital heritage and archaeology. Our 
general workflow has proven useful in many areas of our work (Figure 1). These are then 
transformed into videos, virtual reality, architectural drawings, orthophotos, and virtual 
reconstructions. Lastly, these are served on the web as raw data, as research tools, as sources 
of scientific analysis, and as the art of archaeology and architecture. Resulting from this 
work, we present three brief examples. 


Terrestrial Photogrammerty Workflow 


Features selected for photo capturing 


Features photographed from multiple 
positions and angles 


Images downloaded, sorted, and organized 
by feature 


Images uploaded by feature to Reality 
Capture software for 3D alignment and 
mesh generation 


3D mesh decimated and exported as .obj 
files 
3D meshes processed, repaired, and 
cleaned in Geomagic Studio 


Repaired 3D meshes imported back to 
RealityCapture for retexturing 


Laser Scanning Workflow 


Reconnaissance of building or other features 
completed 


3D scanning plan developed 


| 


Building 3D scanned with two FARO Focus3D 
scanners 


| 


Data filtered, registered, and colorized in 
FARO SCENE software 


| 


Data converted to octree database structure 
and features exported out of FARO SCENE 
software in sections as .pts files 


Exported data filtered, manually cleaned, 
reviewed in CloudCompare 


Aerial Photogrammerty Workflow 


Aerial extent selected for photo capturing 


Site or building photographed with grid 
and point-of-interest 


Images downloaded, sorted, and organized 
by flight and area 


Images uploaded by feature to Reality 
Capture software for 3D alignment and 
mesh generation 


3D mesh decimated and exported as .obj 
files 


3D meshes processed, repaired, and 
cleaned in Geomagic Studio 


Repaired 3D meshes imported back to 
RealityCapture for retexturing 


Retextured 3D mesh exported as .obj file 
and merged with 3D modeled features 


Data exported for upload to Sketchfab and 
Potree; point clouds exported for orthotiff and 
3D animation creation 


Figure 1. the Global Digital Heritage workflow. 
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Retextured 3D mesh exported as .obj file 
and merged with 3D modeled features 


The first is the integration of terrestrial laser scanning and aerial photogrammetry at the 
Castillo de Los Vélez, in Mula, Murcia, Spain. This is a late Medieval castle built on the 
ruins of a massive Islamic fortification. Our goal here was to create an accurate model for the 
community of Mula so that they could begin the restoration and make plans for making the 
castle accessible to the public. This project required the integration of 227 terrestrial laser 
scans of the interior, and 2900 photos of the exterior to create the integrated model of the 
complete structure (Figure 2). 


Figure 2. The exterior photogrammetric model done from aerial images (top), and the interior terrestrial laser point cloud 
(lower) done with Faro laser scanners. 
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Our second example is the Medieval castle of Alhama, also in Murcia, Spain. While 108 
scans and 2450 aerial photos were done here in order to create a result much like the site in 
Mula discussed above, here we used an additional technology. DStretch is a rapid image 
enhancement software used for paintings, originally designed to detect and document rock art 
[5,6]. In this example, high resolution photos were taken of poorly visible paintings on the 
walls of the Medieval tower. Using DStretch, these paintings become vivid and accessible to 
researchers (Figure 3). 


Figure 3. Christian art on the walls of the tower of the Castillo de 
Alhama, in Alhama de Murcia, Spain. DStretch was used to enhance 
the digital images . 
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In a third example, we are scanning Roman archaeological features within the Museo 
Arqueológico Los Baños in Alhama de Murcia, Spain. In this example we scanned an 
archaeological excavation preserved in the museum, and then completed the 3D model. This 
is an example of fine-tuning the output in order to create a powerful visualization of the 
features, and then upload the data to online presentation (Figure 4). 


Figure 4. Figure 4: Model creation for a portion of the Roman 
Bath in Museo Arqueológico Los Baños in Alhama de Murcia, 
Spain. Top — wire-frame mesh; center — un-textured mesh; 
lower — textured mesh. 
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SUMMARY 


Global Digital Heritage (GDH) is a not-for-profit, private research and education 
organization dedicated to documenting, monitoring, and preserving our global cultural and 
natural heritage. We use digital visualization, 3D virtualization, geospatial informatics, and 
open access solutions to provide digital data and 3D models to governments, regional 
institutions, museums, local scholars, and the public. A key element of our mission is the 
democratization of science-we make all data freely available to the world in support of 
cultural heritage, heritage management, education, public access, scientific research, and to 
support of the digital humanities. 


We believe that many nations, local and regional museums, and universities have specific and 
spectacular places, monuments, and museum collections that are critical to the global 
scientific agenda. While these are often recognized for their heritage value, they are 
underutilized in science and research because of distance or because the scientific community 
does not know they exist. 


We use virtualization technology to digitize entire collections, entire museums, and entire 
archaeological and paleontological landscapes. We make virtual repositories available to any 
student, any child, any scientist or any enthusiast anywhere in the world at any time. We 
create online analytical tools to democratize education and research through global analyses 
and exploration. Our virtual repository approach allows for the scientific analyses of places, 
monuments, and collections on a global scale, and provides a means to highlight the 
importance of those collections to their communities. 


In the face of an increasingly hostile world, a global landscape where conflict and natural 
disaster are destroying our shared heritage at an accelerating rate, we provide 3D digital 
services to document and preserve places and specimens critical to our global heritage. We 
do this for free. We then return all of the data and results to the host institution or 
regional/local authorities who can do anything they wish with the materials. 
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Abstract — Real and virtual are often considered terms in reciprocal opposition, but the boundaries 
between the two are blurred. The main goal of our study consists in answering the question whether 
the presence of a third dimension (3D) is a fundamental step of the virtual toward the real world, and 
if it causes some difference in the neural activity of the spectator [8]. Also, the possibility to consider 
real what is virtual will be discussed [6, 7]. 


INTRODUCTION: AIMS AND MOTIVATION 


“Real” and “virtual” are often used as opposed terms but, even if they are still divisible, the 
boundaries between the two are blurred. The properties of reality that cannot be reproduced by the 
virtual world are increasingly diminishing, in a path that will lead the virtual to cross the real and, 
perhaps, to overtake it. 

The term “virtual” has been used since ancient ages and derives from the word “virtue”: by 
definition, it is something that could exist but has not yet been realized, but which has the virtue of 
being realized. Lately, the term was adopted by scholastic philosophy to indicate what exists in 
power and is able to actualize (for example, the tree is virtually present in the seed). The reference 
here is to the Aristotelian philosophy, where one of the categories of being, that of change, was 
related to the passage between potentiality and act [1, 13]. 

Now, we are witnessing the reverse process: the virtualization of an object as the consequence of 
a creative process in which a piece of reality is transformed and coded by means of electronic 
systems. This way, the starting object is still recognizable, but transformed in its natures, being 
recoded in a secondary format. For instance, a chair placed in front of us, but displayed by a 
monitor, is no longer made of wood, but of numbers; it has the appearance of something you can sit 
on, but you cannot interact with, you can only watch it and think about its cognitive and behavioral 
potentialities [11]. Indeed, to understand the basic principles of how 3D technology works, it was 
necessary to understand how vision can create the effect of depth that allows us to locate objects in 
three dimensions. This is attributable to the specific anatomy of vision: in fact, humans have two 
front eyes with an interpupillary distance of about 65mm. Then, the brain operates on the two 
different images, to elaborate a coherent and unitary interpretation of what a person is actually 
seeing. 

The history of 3D technology has a remote beginning, still linked to photography. An example 
was already present at the Great Exhibition of 1851: by using a stereoscope (an invention by D. 
Brewster), a photo of Queen Victoria was shown in 3D instead of the conventional 2D format of 
pictures and paintings. Here, something more than a simple perspective effect was presented: it was 
the real recreation of human vision. 

The first film projected was “The Power of Love” that exploited a traditional strategy to create 
the 3D experience. Indeed, two images were superimposed on the screen, so that they become 
staggered with a “double” effect. Then, special glasses must be used to divide the images. In fact, 
the problem is to make the individual eyes perceive only one image as it was in the old stereoscope. 
At the beginning, this phenomenon was obtained with the glasses, called anaglyphs, with a green 
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and a red lens (then cyan and red for better performance). In fact, they allowed the eye covered by 
the red lens to see only the projection that was not excluded by the filtering of the colored lens. 

Obviously, the notable disadvantage of this technology was the very poor color rendering. At 
present, the most widespread alternative, thanks to effectiveness and low cost, are polarized light 
glasses: this technology exploits the polarization of light to separate the image to be sent to the right 
eye from that meant for the left eye. The lenses are orthogonally meshed (one vertical or one 
horizontal) and the image is filtered and received independently from the two eyes. 

The major flaw of this technology is that the two images are projected simultaneously on the 
screen and must therefore share the resolution. In other words, the final image will have a resolution 
equal to half of the projected one. In addition, some subjects experience headaches of varying 
intensity after a prolonged vision. Finally, it is necessary to keep the head straight; otherwise, the 
retina of the lenses would no longer coincide with the polarization of the image. Indeed, if the 
images are not correctly filtered, an annoying “double effect” occurs. 

Also, there are active glasses that, through an internal battery, perform the required processes to 
dissociate the images of the screen. They are called “shutter glasses” because they are based on the 
same shutter principle of cameras. The lenses of these glasses are composed of liquid crystals that 
alternately blur at a very high speed (up to 300 times per second); the image projected on the screen 
is synchronized with the frequency of lenses opening and closing, in order to send a specific signal 
to each eye. 

In any case, the virtual 3D has now “invaded” the space of the real 3D: what we see is no longer 
limited to that fictitious space that is the monitor or the cinema screen, but moves towards the 
viewer, thus acquiring a spatial dimension [16]. In fact, when we watch a 3D movie, the effect of 
depth is not only created towards the “inside” of the screen (like a perspective effect of a 
Renaissance painting), but some objects extend completely outside the plane to which the screen 
belongs [2, 3]. 

This kind of images also become solid in a geometric way. The viewer of this three-dimensional 
projection has the feeling that the object is within reach. When the visual involvement is optimal, he 
reacts as if he could touch and interact with the object [4]. 

A feature of real objects that is lost in their transposition into two dimensions is corporeity, 
which is the perception of the object observed as an opaque body with its own mass. One thing is to 
see the 2D image of a stone, another to see that same stone in 3D. In both cases, the object is 
familiar and therefore we could attribute it an estimate of weight. However, being in front of this 
object in three dimensions gives us additional information that, with a standard format, we would 
not have. Certainly, not everyone (neither the spectators, nor the directors) appreciates this increase 
in information and the related change in perception, perhaps because the main purpose of a movie is 
not to interact with objects and/or people, but to be captured by a story in a fantastic world. 


OBSERVATION AND BRAIN ACTIVATION 


The observation of an action performed by others triggers in the observer the same neural 
networks, as if we were in the first person to carry out the action. This evidence paved the way for 
the idea that action observation could be used as an effective way to learn or improve the 
performance of a specific motor skill. A particular type of “observation” was already successfully 
applied in the world of sport and rehabilitation. In fact, by using motor imagination, it is possible to 
visualize the movements and involve the motor areas that will then be activated during the 
execution of the same movement [9, 12]. 

However, this requires a high level of concentration on the body part of the subject. The person 
must imagine himself performing specific actions and therefore its application could be difficult, 
especially when the motor imagination is applied for clinical purposes. In fact, it is not always 
possible to ask patients, often in suffering conditions, to carry out this concentration effort. It is also 
impossible for therapists to verify if the subject is visualizing the movements in the correct way and 
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accompanying them in the rehabilitation process. The observation of actions seems a simpler and 
more easily applicable method. 

A recent study by Gatti and colleagues [10] highlighted how action observation treatment (AOT) 
is more effective than motor imagery when it comes to learn a complex motor task. The study 
focused on the learning phase of the movement and, although the neural structures involved were 
the same, it is believed that the visuo-motor system is activated in a deeper way when observing the 
action. In fact, the ventral premotor cortex receives visual inputs and it is stimulated by a visual 
signal rather than by the will to visualize movement while it is not present. 

In addition, during action observation, the subject has a model that performs the action correctly. 
Conversely, motor imagination must rely only on his ability to recall exactly the relevant motor 
representation and to mentally develop the movement correctly [5]. The results of the study also 
showed greater accuracy in the execution of the movement by the subjects who had followed the 
AOT program. 

In the present study, we aimed at answering the following questions: Is the use of three- 
dimensionality in video a fundamental step of the virtual towards the real? Does this lead to some 
difference in the level of cortical and peripheral activation of a spectator? Moreover, some possible 
applications of AOT in clinical practice are desirable [15]. 


THE EXPERIMENT: MATERIALS AND METHODS 
Participants 


Twenty man between 20 and 60 years-old participated in the experiment (M=32.6 years). All 
participants were right-handed with normal or corrected-to-normal visual acuity. Exclusion criteria 
were: left-handedness, the presence of any neurological disorder or cognitive impairment, visual 
disorders, a previous surgery at the left knee or pre-existing motor impediments (such as 
hemiparesis, lumbar sciatica). No payment was provided for subjects' performance and they gave 
informed written consent to participate in the study. 


Stimuli and procedure 


After taking place on a comfortable chair in a dimly lit room, two videos, identical in their 
contents, were proposed to participants, both in 2D and 3D presentation. Videos comprised a target 
condition, in which leg movements were displayed, together with two control conditions, in which 
arm movements or naturalistic neutral scenes (N) were proposed. In detail, the extension movement 
of the left leg (L) and the flexion movement of the left arm (A) performed by a male actor were 
used. Each different content was presented four times, in a randomized order. The video cuts lasted 
one minute, and were assembled in randomized sequences, for a total of 12 minutes for each 
modality (2D vs. 3D): four minutes of G, four minutes of B, and four minutes for N. The order of 
vision of 2D and 3D movies was also random (see Fig. 1). The videos have been filmed with 
constant artificial light (280 lux). During the experimental phase, changes in the muscular tone 
(electromyography, EMG) and in the cortical activity (EEG) were measured. 
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1 min 1 min 1 min 


LEG NEUTRAL ARM 


Fig. 1: Display of stimuli presentation with Leg (L), Arm (A), and Neutral (N) conditions presented 
in a randomized order. 


EEG recording 


EEG signal was acquired by using a MindCap XL Headband (Neurosky). The tool consists of an 
elastic band with due electrodes: the active one was positioned over Fpl, and the reference 
electrode over Fp2. Also, a clip to be attached on the left earlobe was present. The system was 
battery powered and the signals were sent via Bluetooth to a computer that recorded them through 
the NeuroView software. The sampling frequency was 512 Hz for the filtered signal and 7 Hz for 
the Power Spectrum reporting the frequency of all brain waves with a range of 4 Hz (Delta 0-4 Hz, 
Theta 4-8 Hz, Alpha 8- 12 Hz, Beta 18-22 Hz, GammaLow 28-32 Hz, GammaHigh 38-42 Hz) (see 
Fig. 2). 


Fig. 2: MindCap XL Headband positioned over Fpl. 
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EMG recording 


Muscular activation involves the action of muscles and nerves, which is triggered by very small 
electrical currents. Measuring the electrical activity in muscles and nerves can be useful for Human- 
Computer Interaction, control, biofeedback and many other applications. The sensor is capable of 
performing electromyography (EMG) measurements using bipolar surface electrodes (plus a ground 
lead), and monitors the muscle activation. 

EMG signal was acquired by using BITalino (Plux Wireless BioSignals SA) and OpenSignals as 
acquisition software. BITalino is wireless bluetooth device that can collect different biosignals. The 
EMG signal was collected with a sampling rate was 250. 


RESULTS 


The power of each EEG band was submitted to different repeated-measures ANOVAs with 
video content (3 levels: L, A, and N) and modality (2 levels: 2D, 3D) as repeated factors. A 
significant Content effect has been found with respect to Alpha waves (F1,2=6.04; p<0.01). Post- 
hoc comparisons showed that Alpha levels during the observation of leg movements were 
significantly lower compared to arm motion videos and neutral natural scenes. No significant 
differences were found between arm and neutral scenes (Fig. 3). 


Alpha Power (normalized) by Condition 


Fig. 3: Alpha Power (normalized values) while viewing Leg (left), Arm (centre), and Neutral (right) 


movies. 


No significant differences were found with respect to the projection mode (2D and 3D) even if 
constant trends were found: during 3D projections higher power values were recorded, than those 
found in the 2D video. Moreover, no significant differences were found for peripheral measures. 


DISCUSSION AND CONCLUSION 


The present work aimed at exploring the brain and peripheral correlates during the vision of 2D 
vs. 3D scenarios depicting specific body movements with rehabilitative purposes. 
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A significant result was found for video content in the alpha range, with lower values during 
observation than arm and neutral. Alpha waves are typically linked to the functioning of the medial 
cortex. The areas that lie between the prefrontal areas up to the parietal lobe constitute the so-called 
“default system”, and are intended to manage the cognitive system in the absence of a particular 
high level engagement. Thus, when a subject is engaged in a task that is not particularly 
challenging, these areas become dominant and lower the frequency of the areas that are not 
involved in the current task. On the other hand, when there is a relevant cognitive task, this default 
system loses its synchronization value and the various structures related to the task raise. This 
interpretation is supported by a study conducted by Mo and colleagues [14] in which the activity of 
Alpha waves was recorded for short intermittent periods with eyes closed and open showing how, 
on average, the intensity of the oscillations of Alpha (8-12 Hz) decrease in the tasks in which 
attention is focused on the outside, as could be the observation of images. 
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Introduction 

The attribution of great paintings is one of the most important activities in modern museum work. The attribution of 
an artwork is the result of authentication — a procedure for the confirmation of its authenticity. The main task in 
authentication is to find material evidence by which it is possible to indisputably acknowledge the experts’ 
conclusions about the authorship and the painting’s time of creation. The author’s signature, the dates and 
monograms left by artists on the front or back of the picture, as well as accompanying inscriptions and historical 
information on the provenance of the work are among such proofs. Unfortunately, the influence of the environment, 
adverse storage conditions and the consequences of radical restoration intervene in the structure of the painting and 
often lead to these proofs disappearing. 

Furthermore, in many paintings, the artist’s signature is actually absent. In such situations, it is necessary, 
in order to remain objective, to use natural-scientific approaches, including various optical and physical methods for 
studying the structure of the painting. However, despite the availability of a wide range of contemporary analytical 
research methods, they do not always prove to be effective enough to obtain the necessary information. In fact, the 
determination of the composition of the paint layers and the dating of a painting’s support (by means of carbon 
dating and dendrochronological analyses), even combined with research results by art history experts, does not 
always allow for its attribution to the hand of a certain artist. Hence, the need to develop a new scientific method for 
studying paintings, which provides information that cannot be obtained with the use of traditional research methods, 
and which avoids subjective opinion. 

In recent years, numerous computer methods based on the analysis of optical images obtained with the aid 
of digital photography have appeared in the field of the authentication of artworks. Most methods of computer 
analysis serve as a means for studying the visual characteristics of an artist’s brushstrokes. Till now this information 
has been used to study paint layers and style. However, since in most cases the technique of applying the paint is 
highly individual for each artist, it can be considered as a “fingerprint”. How to extract and analyse this “fingerprint”? 
The mathematical methods of processing a digital picture (“quantification”) include the statistical method of a 
wavelet analysis, the support vector method, the fuzzy clustering method, etc., which are all used to analyze the 
brushwork technique [1] — [4]. In some cases, these methods are combined with analysis methods from other 
scientific fields, e.g. biometrics and medicine [5]. In this article, a scientific method for the comparative analysis of 
paintings is described, which employs intensity histograms of their digital optical images being the extraction of this 
“fingerprint”. The possibilities provided by this method are illustrated by a case study of two paintings by Leonardo 
da Vinci. 


Basic principles of histogram analyses in paintings 
When creating a picture, an artist often applies a huge number of brushstrokes to a canvas , a panel or another 


support. The resulting painting surface thus acquires specific spatial and spectral characteristics that can be 
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identified for the analysis of the individual painting technique of the particular master. The brushstrokes of each 
artist, as a rule, have a certain length and direction, as well as a viscosity of the paint particular to this master. They 
result from the texture of the brushes, the speed of the hand movements, the particular features of the color palette, 
as well as from the pigment mixing techniques and the use of glazing. 

In light of this, one of the possible ways of studying the properties of an artist’s painting technique is to 
analyze the histograms from digital images of the paintings. Any amplitude histogram (another name for the 
intensity histogram) of a digital optical image is a function (graph) of the statistical distribution of the image 
elements of various intensities, in which the horizontal axis indicates the brightness level and where the relative 
number of pixels with a specified brightness value are plotted on the vertical axis (a typical histogram is shown in 


Figure 1). 


Mean: 165,87 Level: 
Std Dew: 45,89 Count: 
Median: 176 Percentile: 
^ Pixels: 480000 Cache Level: 1 


Fig. 1. *Young Woman with Unicorn" by Raphael (c. 1505-1506; Galleria Borghese in Rome) 


and histogram of digitized image of face of woman 


The histograms from the digital images of paintings can be considered as the artists" "fingerprints", as 
mentioned above. This is connected with the fact that the histograms display a unique distribution of light and dark 
shading reflecting the style of each master. In fact, the analysis of pictures with the use of histograms is a 
quantification of their chiaroscuro, meaning "light and darkness", and it refers to the technique of painting that is 
characterized by a sharp contrast of light and dark tones. This technique serves to convey contrasts, often quite 
dramatic, in a painting. The term chiaroscuro should also be understood as the general character of the distribution 
of light and dark tones throughout any pictorial image. In the opinion of many art historians (including K. Clark and 
C. Pedretti [6], [7]), the analysis of the chiaroscuro feature plays the most important role when comparing the 
paintings of different great masters, since in the process of creating the works each artist develops the dance of 
"shadows" that is peculiar to him, and which is a unique individual characteristic of his painting technique. Thus the 
scientifically extracted histogram of the digital image of a painting is the identity of an individual artist's 
"fingerprint". 

As a rule, chiaroscuro is a result of the blending of the color saturation and albedo from one region of a 
painting to the adjacent zones. Under the traditional subjective approach of the attribution of paintings, the art expert 
evaluates a picture visually (without any use of analytical methods). The properties of the spatial blending and 
contrasts of light and shade are the main features that are taken into consideration to establish the authorship of the 
artist's works and to determine forgeries or copies. This visual impression is correlated in the expert's mind with his 
memory of impressions of similar authenticated works by the same artist. As a result of this comparison, the expert 
gives his/her conclusion. Obviously, the analysis of the intensity histogram of the digital image of the picture, will 


allow the experts to conduct a more objective examination with the aid of mathematically precise data. 
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Method of the analysis of pictures through intensity histograms 

Regarding the technical aspects of a histogram analysis, in all the typical formats of digital optical images, the pixels 
range over 256 intensity levels (their values can vary from 0 to 255). Currently, many computer digital image- 
processing programs incorporate special functions that allow us to obtain reliable histograms. Using the histogram 
“intensity” option, we can then count and plot the number of pixels (in the whole image or an individual region) for 
each of the 256 intensity levels. The graphical dependence of the quantification (the number of pixels) on the 
intensity level is known as an amplitude or intensity histogram. 

When paint pigments are blended uniformly, the luminosity of the picture image can vary widely (from 
bright-light to very dark tones). In this case, the gradient in the luminosity distribution of the individual pixels from 
the highest to the lowest intensity values will be uniform (there will be no discontinuities or abrupt transitions in the 
distribution of the histogram). It should be added that each intensity histogram is characterized by two basic 
mathematical parameters: mathematical mean and standard (mean-square) deviation. When analyzing the artists” 
brushstrokes, the most important characteristic from these two parameters is the standard deviation. Therefore, when 
a painting is being examined by means of an analysis of the histogram of its digital image, one should not only 
compare the form of the distribution, but also take into account the value of this standard deviation. Paintings 
created by the same artist should obviously have a close similarity in terms of the painting technique, and 
consequently, their histograms should be similar to each other according to these two criteria. This is key to a 
comparative analysis of paintings based on a quantitative exposure of the artist”s style through amplitude histograms. 

In order to obtain more precise results when comparing histograms, it is advisable to convert the color 
image to a grayscale (that is, to a black and white image). The results can be more accurate if a comparative analysis 
of the histograms from color images is carried out in different spectral bands (separately in the red, blue, and green 
ranges). 

It should be understood that histogram analysis is specifically a method of comparative analysis, i.e. 
comparing the painting under examination to others which have already accepted attributions. It is important to bear 
in mind that the painting techniques can change throughout an artist’s life. Therefore, some differences in the 
characteristics revealed through histograms can be explained by the evolution of the artist’s style and technique. It is 
also important to take into account that artists with little or no chiaroscuro technique may not manifest themselves in 
the individual histogram features of the digital images of their pictures. 

As shown in research studies [8], [9], the works of those masters with a very delicate painting technique 
most often have very distinctive features in the histograms. An excellent example is Leonardo da Vinci. Leonardo is 
well known for being the first to apply the technique of sfumato (from the Italian sfumare, meaning “to evaporate 
like smoke”). This term denotes a manner of painting that is marked by fine shading which softens the outlines of 
figures and objects, and even allows the artist to convey the air that envelopes them. It was discovered in the course 
of recent research involving Leonardo’s paintings that this effect was achieved by applying brushstrokes with a 
thickness of several micrometres, and that the total thickness of the paint layer often did not exceed 30-40 
micrometres. 

It is important to note that in portrait painting, analyses of histograms are the most effective when 
comparing facial characteristics. The subject of such a histogram analysis is best as a comparison of the parts of the 
paintings depicting human faces, for the face is the key element in the composition of a portrait. It is known that 
great masters often involved their assistants and apprentices in the performance of their work. However, the 


depictions of the faces in portraits were, as a rule, executed by the masters themselves. Consequently, a histogram 
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analysis of the facial images is the most reliable way to establish if works being compared belong to the hand of the 


same master. 


Comparative analysis of the Leonardo paintings “Mona Lisa” and “Isleworth Mona Lisa” 
As a clear illustration of what can be achieved by the histogram method, we present the results of a comparative 
analysis of the paintings Mona Lisa” and “Isleworth Mona Lisa”. The composition of the “Isleworth Mona Lisa" is 


very similar to the famous “La Gioconda” displayed in the Louvre (Figure 2). According to many painting experts 


[10], [11], it is an earlier version of this subject, and it therefore is also often referred to as the “Earlier Mona Lisa”. 


Fig. 2. Left: Louvre “Mona Lisa”; Center: “Isleworth Mona Lisa”; Right: “Young Woman" (c. 1504) by Raphael 


Like most Leonardo paintings, the certainty of its early provenance is not absolute, but historical documents 
lead us to conclude the existence of two original Mona Lisa paintings in the early 16" century — one in the French 
Royal Collection now in the Louvre Museum, and one with side columns that Raphael copied in a 1504 sketch (Figure 
2) which was listed in the estate of his servant Salai. That latter painting is acknowledged by many experts today as the 
“Isleworth” or “Earlier Mona Lisa”. This painting also fits Vasari's description of being unfinished and must be the 
seminal work which generated all the copies we have today with columns. 

Most of the important detailed scientific research of the “Isleworth Mona Lisa” (including X-rays, UV- and 
IR-ray examinations, studies of the pigments with the aid of spectral methods, radiocarbon analysis and others (see 
the details in [12])) has been carried out during the last 10 years. 

From this research, it was found out that all the pigments and the ground of the picture are identical to the 
materials used in the early 16° century by Italian artists and by Leonardo himself. Besides this, following carbon 
dating tests of the support, the approximate time of the creation of painting was determined to be between 1492 and 
1652 with highest probability between 1493 and 1525. New studies were carried out by the authors of this article 
with the aid of a comparative analysis of the digital images of both paintings. 

In 1989, following his extensive and successful work on the attributions of Rembrandt self-portraits, on the 
instigation of Dr. Kenneth Clark, one of the authors (John Asmus) received a special invitation to study in depth the 
Louvre “Mona Lisa” including a complete day alone with the famous artwork. He carried out a computer 
“restoration” of the Louvre “Mona Lisa”, which revealed an idea of the original appearance of the portrait — without 


the dark yellow varnish and the craquelure webs that lie over the image [14]. This was soon followed by an 
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invitation to examine the “Isleworth Mona Lisa”. As the results of these studies Asmus became widely known as a 
top scientific expert on brushtroke technique and identification. 

During the first stage of the research on the detailed studies of digital images of both paintings and their 
comparisons were conducted. The proportions and arrangement of the key elements in the composition of both 
paintings showed that the “Earlier Mona Lisa” could not be a copy of the Louvre painting since the pictures differ 
significantly from each other in terms of the proportions of the images and in many other fundamental matters of 
construction. In addition, their main axes (the axes that can be aligned along the eye line (horizontally) and the 
hairlines section (vertically) do not coincide (see Figure 3) (for details, see [15]). When comparing the intensity 
histograms from the images of both paintings the very first results proved to be astonishing. The character of the 
distribution and the main parameters of the histograms (mathematical mean and standard deviation) in the areas of 
the face were almost identical in both pictures (Figure 4). Consequently, if the histograms of the digital images are 
the “fingerprints” of their authors as proven in the experience with the Rembrandt portraits, we can draw the 
conclusion on the basis of the experimental data that the faces of the “Isleworth Mona Lisa” and the Louvre “Mona 


Lisa” would most likely have been created by the same artist. 


Fig. 3. Comparison of geometrical features of images of the Louvre “Mona Lisa” (left column) and “Isleworth Mona Lisa” 


(right column) 


—. att 


Fig.4. Histograms of digitised images of the Louvre “Mona Lisa” (left) and “Isleworth Mona Lisa” (right) 


To give such a conclusion a more solid ground, supportive analysis of the histograms of digital images from 
the most famous copies of “Mona Lisa” was undertaken. These include: 1. The “Mona Lisa” from the Prado 
Museum in Madrid (Spain) considered to be the earliest copy of the Louvre “Mona Lisa”; 2. The “Mona Lisa” from 


the Museum in Oslo (Norway) dating from the 17" Century; 3. The “Mona Lisa” from the Walters Art museum 
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(USA), attributed to Simon Vouet (1590-1649); 4. The Reynolds “Mona Lisa” (private collection, UK) and 5. The 
“Mona Lisa” Flemish school (private collection, UK) dating from the XVI and XVII centuries, respectively, 

All of the portraits listed here are shown in Figure 5. The study of the histograms of their digital images 
showed that they all have significant differences that make them distinct from the histogram of the Louvre “Mona 


Lisa” (Figure 6 shows some of the histogram distribution characteristics). 


Fig. 5. From left: Prado “Mona Lisa”, Walters “Mona Lisa”, Flemish “Mona Lisa”, Reynolds “Mona Lisa” and Oslo “Mona 
Lisa” 


In the data comparison shown in Figures 4 and 6, it is demonstrated that the histograms of the digital 
images of all the copies are distinctly different from the histograms of both the Louvre “Mona Lisa” and the 
“Isleworth Mona Lisa” (they differ not only in the distribution characteristics of the histograms, but also in their 
basic parameters). This significantly distinguishes the “Isleworth Mona Lisa” from the copies of “Mona Lisa”. 
However, in order for the attribution to be even more accurate, we also compared the histograms of the individual 


features of the faces in all the portraits (eyebrows, eyes, nose, mouth, chin, etc.). 


c d 


Fig. 6. Histograms of digitised images of the Louvre Mona Lisa (a) and its copies: with The National Gallery (Oslo) (b), 


with The Walters Art museum (c) and with Prado museum (d) 


According to Morelli [16], studying the details of the art form allows us to reveal the specific character of 
the individual style of each master. Morelli was convinced that the artist's personality manifested itself in small 
details, since both the artist and the imitator displayed the individual features of their painting technique by a natural 
stroke, rather than in a neat signature which is applied consciously in both cases. This is due to the fact that while 
painting the details, the artist becomes relaxed and acts intuitively. Consequently, an analysis of the individual parts 


in paintings with a similar composition may reveal the distinctions in the techniques of different artists most clearly. 
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In addition, a comparison of the histograms of the images of all the above-mentioned works was carried out 


in different spectral bands (in the blue, green, and red ranges). Some of these histograms are shown in Figure 8. 


a b c 


Fig. 7. Histograms of digitised images of Reynolds Mona Lisa (a), Louvre Mona Lisa (5) and “Isleworth Mona Lisa" (c) 


As can be seen from the graphs in Figures 4 and 7, in all cases, there is a very good correlation between the 
histograms of the “Isleworth Mona Lisa” and those of the Louvre “Mona Lisa”. This similarity becomes particularly 


remarkable if the histograms of these two portraits are superimposed on each other (see Figure 9). 


Fig. 8. Result of matching of histograms of digitized images of faces (in the field of nose) appeared on paintings 


“Isleworth Mona Lisa” and Louvre Mona Lisa (background is histogram of “Islweworth Mona Lisa") 


On the contrary, the comparisons of the histograms from the Louvre “La Gioconda” and its copies in similar 
studies indicate many apparent differences. The same result is demonstrated in a 3D-diagram (Figure 9). It shows 
the normalized values of the standard deviation of the histograms of the Louvre “Mona Lisa” (LML), the “Isleworth 
Mona Lisa" (EML), the copies from the Prado museums and the Oslo museum, and the copy executed by the 
Flemish painter. On the three axes in the diagram, the values of the standard deviation of the histograms from all 
these pictures in the regions of the nose, mouth and eyes are plotted. The graph clearly displays that the Louvre “La 


Gioconda” and the “Isleworth Mona Lisa" have the best correspondence in the standard deviation. 
Um 


Fig. 9. 3-D plot of the statistical variances of three regions (eyes, noses, and mouths) of five Mona Lisa 
paintings (Louvre and Earlier/Isleworth versions as well as the Prado, Oslo, and Flemish copies) 


Thus, based on the results of careful retracted studies of digital optical images of the Isleworth and Louvre 


“Mona Lisa” paintings, we can conclude that the faces in both works were created by the same artist. 
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Conclusion 
In this article, a computer method for the comparative analysis of paintings is described, which could be useful for 
making the final decision on the attribution of paintings in those controversial situations when traditional methods of 
examination do not work or where there is a disparity in the subjective opinions of experts. It is worth mentioning 
that this approach was recently applied by the authors to the study of Rembrandt's self-portraits, where it showed the 
highest degree of efficacy. The results of that work will be published in a separate article, but it should be noted here 
that there is a clear correspondence between the results of the authors” research and the data on the attribution of 
Rembrandt’s self-portraits, which were obtained in his paintings with the aid of technical and technological studies, 
and by an art examination conducted within the framework of the international Rembrandt Research Project [17]. 
Finally, it is important to say that the success and the further progress of this approach will depend entirely 
on the creation of an extensive histogram database of the properties and statistics of the most famous artists? works, 
which are recorded as a result of detailed studies. This may lead to more precise conclusions about the final 
attributions of the paintings on which art historians’ opinions currently differ. It will provide a scientific method 
which may eventually be accepted as the most credible and popular way chosen by museums and collectors for the 


attribution of great artworks in light of its obvious objectivity. 
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Preservation of the cultural heritage in digital form and its presentation in the Internet using 
multimedia and virtual reality technologies is a priority activity of the Center for Design and 
Multimedia of the ITMO University. In recent years, the Center has implemented a number of 
projects in this field with the support of the Foundation for Humanities. 


The projects are aimed at creating a Multimedia Information Systems for various cultural 
monuments of the North-West of Russia. Among them are: “Architectural Ensemble of the 
Solovetsky Monastery in the Period of Its Highest Prosperity (XVI-XVII Centuries)” 
(http://solovky.ifmo.ru), "Architecture and art complex Theodore town in Tsarskoye Selo as an 
example of Russian style" (http:// russianstyle.ifmo.ru /). In each project, multimedia and virtual 
reality technologies to give the most complete representation of the object of cultural heritage. 


In the XVth century the Solovetsky Monastery was the spiritual, cultural and political center of 
the Russian North, playing a crucial part in the emergence of the economy and maritime 
practices of Pomorye region. The Solovetsky Monastery features some remarkable monuments 
of ancient Russian architecture — Assumption Church (1552-1557), Cathedral of Transfiguration 
of Our Savior (1558-1566), gatehouse Annunciation Church (1601). Virtual reconstruction 
provides an opportunity to get acquainted with the original appearance of Solovetsky Monastery 
XVI - XVII centuries. 


Fig. 1. A general view of the Solovetsky Monastery (XVI-XVII Centuries) -3D reconstruction 
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Russky Gorodok was built in Tsarskoye Selo, an imperial residence outside St. Petersburg, in 
1913-1918 by the order of Nicholas II. The main idea was to visualize the history of Russian art 
and architecture in the 12-17" cents., to give the people from the metropolitan, largely European 
city of St. Petersburg an opportunity to appreciate the authentic Russian Style. From the very 
start, Fedorovsky Gorodok was informed by certain virtuality, for the idea was to create an entity 
that had never existed before, yet would absorb the elements of the authentic artistic lexicon on 
one hand, and the most advanced contemporary technologies on the other. 


Fig. 1. A general view of the Russky Gorodok -3D reconstruction 


We used various virtual reality technologies to fully represent reconstructed monuments in 
multimedia information systems. Among them: 

- Virtual excursions via video 360 as in present time and in reconstructed reality. Video 
360? of historical attractions is a unique opportunity for users to see the main sights at any time 
with effect of presence 

- Interactive maps with active points thanks to which the user is able to learn about the 
architecture and have access to virtual tours shooting in format video 360 and additional 
historical information. 

- Video films (in standard format and in video 360 format) telling about the architecture 
and the history of cultural monuments. 

- Interactive 4D-excursions that allow you to combine virtual tours, shot with the help of 
360 video at the present time, with interactive excursions in the historically reconstructed virtual 
reality with the option of moving from 360? video to 3D reconstruction format and back at all the 
key points. 


This work was supported by the project “Multimedia information system "Architecture and Art 


complex Theodore Town in Tsarskoye Selo as an Example of Russian Style" ” (Grant Ne17-04- 
12034, Russian Foundation for Humanities) 
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Abstract 


Biological evolution intertwines with cultural (today mainly 
technological) evolution, whose mechanisms are partly Lamarckian, 
to beget a bio-cultural evolution. Technological tools are produced 
by man to modify the environment and to get information about it. 
In turn instruments exert a feedback on man converting him into 
Homo technologicus, a bio-technological symbiont subjected to a 
continuous transformation. Nowadays such transformation has 
become deliberate and intentional, and aims at two kinds of target: 
therapy and enhancement (to repair or to improve natural 
capabilities or to generate new ones). Enhancement obtained 
through the use of technology opens the post-human perspectives. 
However, man has always been post-human, since he has always 
been hybridized with plants, animals, food, medicines, drugs, and, 
today, machines; and has always been altered and strengthened by 
artificial processes. This continuity makes the post-human concept 
les shocking, since it inserts the post-human into the flow of a 
natural-cultural evolution; on the other hand it bestowes the full 
responsibility of his own evolution upon man. Actually while man 
has always been post-human, only today he is fully aware of this by 
virtue of the overwhelming power and speed of technological 
development. Such novel awareness propounds dramatically the 
serious ethical problems generated by this new progresses. On the 
other hand most post-humanists are more interested in the 
enhancement of cognitive capabilities than in the social and ethical 
aspects. 
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"Cogito ergo sum" (French: Je pense, done je suis; I think, therefore | am), 
Rene Descartes 


ABSTRACT 


The main idea behind this research paper is that modern 
information and communication technology could be better 
made to serve human beings, if we could specify more 
precisely the process of human thought and action. 

The cognitive stages of rational thinking has been studied 
from the user interface and product point of view but there 
does not seem to be any generally accepted model for the 
dynamics involved in cognitive stages in literature. 

In addition, a few studies have investigated the cognitive 
stages in rational thinking process from human centric point 
of view, i.e. how the skills are actually learned. 


INTRODUCTION 


First, I shall provide short historical preview of between 
human thinking, learning and performance. 

Jean Piaget (1896-1980) believed that the process of thinking 
and the intellectual development has two on-going processes: 
assimilation and accommodation. There is assimilation when 
a child responds to a new event in a way that is consistent 
with an existing schema. The schema describes as pattern of 
thought or behavior that organizes categories of information 
and relationships amount them. There is accommodation 
when a child either modifies an existing schema or forms an 
entirely new schema to deal with a new object or event. 
DiMaggio [6]. It seems that the accommodation in modern 
society is becoming more important than assimilation. 

Alan Turning (1912-1954) was pioneer in the development of 
theoretical computer science. The Turing test is a test, 
developed by Alan Turing in 1950, of a machine's ability to 
exhibit intelligent behavior equivalent to, or indistinguishable 
from, that of a human. Turing proposed that a human 
evaluator would judge natural language conversations 
between a human and a machine that is designed to generate 
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human-like responses. 


Human performance by Fitts & Posner (1967) was a textbook 
designed to introduce students to a new field of psychology. 
The field involved the quantitative measures of human 
capacities to perceive, attend, reason and act. 

First, Fitts &Postner (1967) outlined a theory of learning 
involveda) cognitive, b) associate and c) autonomous stages. 
In a cognitive phase during which the performer develops a 
mental picture and fuller understanding of the required action 
to form an executive program, an associative phase during 
which the performer physically practices the executive 
program learned in the cognitive phase and an autonomous 
phase during which the performer learns to carry out the skill 
with little conscious effort. 

Second, Fits, described a quantitative theory of human 
movement control. The time to begin a movement was related 
to the uncertainty of the event and the compatibility of the 
codes relating stimulus to response, which the rate of 
movement was function of the information it generated. 
Stuard Card [4] and his colleagues proposed the first 
stimulation of a user for HCI in 1983 (GOMS), where a 
designer could evaluate an interface by simulating how users 
perceive, think and act when completing tasks. Subsequent 
models (such as ACT-R) extended this modeling to 
consider factors such errors and learning. However models 
become difficult to use and extend. To aid practitioners, 
mathematical simplifications such as KLM and GLEAn and 
interactive modelling environments (like CogTool and 
Distract-R) were developed, but these were not combined with 
algorithms that could generate designs. Oulasvirta [14] 
Nowadays, in the field of cognitive science, researchers has 
focused to study human thoughts using neuroscience i.e. 
neural imaging to discover cognitive stages of rational 
thinking. For instance multi-voxel pattern recognition 
techniques combined with Hidden Markov models has be 
used to discover the mental states that people go through in 
performance a task [1,2]. However, most of the cognitive 
science experiments are still conducted in laboratory settings. 
In the year 2018, Alan Turning test is still failed by all 
intellectual personal assistance such as Apple's Siri, Amazon's 
Echo and Samsung's Bixby. Therefore, to continue Alan 
Turnings work and to develop more human like intellectual 
personal assistance, we need to focus our effort to study more 
deeply the interaction and human thinking models in the 
context of most developed artificial intelligent applications 
(Siri, Echo, Bixby) and robotics such as Watson Pepper. 
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1.1. COGNITIVE MODELS OF LEARNING 


Here I shall present four cognitive models of learning i.e. 
cognitive stages of rational thinking when the human is taking 
a new artifact in use. 

Anderson & Fincham [1,2] introduced the Adaptive Control 
of Thought-Rational (ACT-R), which is a cognitive 
architecture: a theory for simulating and understanding human 
cognition. Researchers working on  ACT-R strive to 
understand how people organize knowledge and produce 
intelligent behavior. As the research continues, ACT-R 
evolves ever closer into a system which can perform the full 
range of human cognitive tasks: capturing in great detail the 
way we perceive, think about, and act on the world. 

Anderson & Fincham [1,2] conducted a study that looked at 
the cognitive stages participants engaged in when solving 
mathematical problems. These stages included encoding, 
planning, solving and response. The study determined how 
much time participants spend in each problem solving stage 
when presented with mathematical problem. Multi-voxel 
pattern recognition analysis and Hidden Markov algorithms 
models were used to determine participants’ problem solving 
stages. The combined method identifies both the mental states 
and how their duration vary with experimental conditions. 
Result of the study showed that the time spend in the planning 
stage was dependent on the novelty of the problem. The time 
spend in the solving stage was dependent on the amount of 
computation needed to produce a solution once a plan is 
devised. Lastly, the time spent in the response stage was 
dependent on the complexity of the response required by the 
problem. 


Encoding + Planning — Solve +» Response 


Figure 1. Cognitive stages with participants solving 
mathematical problems. Anderson & Fincham [1,2] 


Anderson & Fincham [1,2] states that discovery of encoding, 
planning, problem solving, and responding states was not 
surprising and was anticipated in a previous cognitive model 
for the task. However, there were a number of surprising 
aspects of these states not anticipated: 

At the end, according to Anderson & Fincham [1,2] memory 
has the ability to encode, store and recall information. 
Procedural memory, made of productions. Productions 
represent knowledge about how we do things: for instance, 
knowledge about how to drive bicycle. At each moment, an 
internal pattern matcher searches for a production that 
matches the current state of the buffers. Only one such 
production can be executed at a given moment. That 
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production, when executed, can modify the buffers and thus 
change the state of the system i.e. change the behavioral 
model of human being. 

The second Elliott et al. [10] cognitive model of learning 
found that the process of cognitive thinking is linear with the 
following causalities: a) transparency of operations, b) 
transparency of purpose, c) accommodation and d) 
accomplishment. 

In other words, transparency of operations and transparency of 
purpose lead to a sense of accommodation and finally to the 
sense of accomplishment. In addition, the poor transparency 
of operations and purpose lead to increased effort and longer 
task completion times. In other words, transparent design 
minimizes cognitive demand on the users [16] 

The third one is Gagne's et al. [7] cognitive model of learning, 
which includes: the motivation (expectancy), apprentice 
(attention, selective perception), acquisition (coding, storage 
entry), retention (memory storage), recall (retrieval), 
generalization (transfer), performance (responding) and 
feedback (reinforcement) phases. 

Gagne's et al. [7] model of learning is described in 
psychological and cognitive science point of view. Gagne's et 
al. [7] model of learning is linear and it emphasis the role of 
motivation in learning process. The users expectancy i.e. 
presumptions toward the task or artifact is seen important 
element, which effect positively or negatively the other phases 
of learning. 

The forth cognitive model of learning is presented by 
Laakkonen (2007) involve six phases: 1) information 
search, 2) data collection, 3) knowledge management, 4) 
knowledge form, 5) knowledge build and 6) result of action. 
In learnability perspective: information search and data 
collection phases are the most demanding and most time 
consuming, when taken the new technological artifact in use. 
In the table 1 the four cognitive models of learning is 
presented and phases of learning are compared. 


Table 1. Cognitive models of learning (phases) by Gagne et. 
al [7], Elliott et al. [10], Laakkonen [12] and Anderson & 
Finchman [1,2] 


agne et al. [7] Elliott et al. [10] Laakkonen [12] nderson & 
inchman [1,2] 
eception (gaining transparency of 
ttention) loperation 


xpectancy (motivation, 
earning objectives) 
etrieval (recall of prior ncoding 
learning) 
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selective perception transparency of [Information search 
(apprentice, presenting purpose 
the stimulus) 
sematic Data collection lanning 
encoding/storage entry 
(acquisition, providing 
learning guidance) 


responding (eliciting accommodation Knowledge Solving 
performance) management 

reinforcement Knowledge form 

(providing feedback) 

retrieval (assessing Knowledge build 
performance) 

retention (memory 

storage) 

transfer (enhancing accomplishment [Result of action esponse 


generalization) 


In Elliott et al. [10] model the transparency of operations is 
directly related to the efficiency of a user interface that allows 
users to find, understand and then use rapidly and 

easily the functions of the user interface to complete a task or 
sub-task. In addition, transparency of operations refers to 
concept guessability used by Dix et al. [5], Bruijn et al. [3]. 
They define guessability as an indication of intuitiveness, i.e. 
how obvious the operations are that can be performed by users 
who have no experience with the device and have not received 
any earlier instructions. Bruijn et al. [3] use the term 
guessability as synonym for learnability. 


Transparency of purpose means that users should be able to 
imagine the end product at any point during its use. However, 
it would be beneficial if transparency of purpose is understood 
and seen before the interaction process and not during it. The 
third phase of cognitive model of learning, accommodation is 
more related to the concept of easy-to-use than it is to easy-to- 
learn. The forth phase is very close to the concept of 
usefulness, which is separate concept from that of learning. 


What are the differences and similarities between Anderson & 
Fitchman (2013) structure of thoughts and Laakkonen [12] 
theoretical model of learnability? The cognitive model of 
learning is non-linear and learning dynamics occur inside the 
six phases. The dynamic means that in every phases of 
learning has their own dynamics. For instance in information 
search phase user is moving towards and away from the right 
solution i.e. the process is not linear it is circulated and 
iterative. In addition, the six phases do no follow any given 
path. 


Information search and data collection phases by Laakkonen 
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[12] can be related to Anderson & Fitchman (2013) encoding 
phase. Knowledge management is related to Anderson & 
Fitchman (2013) planning phase; however planning phase 
occurs already before the new artifact are taken in use. 
Knowledge form and knowledge build phases are related 
solving phase and result of action refers to Anderson & 
Fitchman's model's response phase. 

In addition, it needs to be emphasis that knowledge 
management phase [12] needs to be investigate more detailed, 
because in that phase new knowledge is implemented to 
internal patterns "buffers" of human mind. Knowledge form 
and knowledge build phases are related to productions system 
and result of action refers to "change the style of system i.e. 
change the human behavior model. (see Anderson & Fitchman 
[1.2] and Laakkonen [12] 


At the end, the theory by Fitts & Postner [8] emphasizes that a 
user has to "know what" before "know how" when interacting 
with the user interface. In addition, the assumptions of human 
towards a program are very close to Gagne's motivation 
(expectancy) phase of learning [10]. 


CONCLUSIONS 


The information technology key research areas such as: digital 
health technologies, artificial intelligent, big data, internet of 
things, block chains, autonomous driving, robotics, 
augmented reality, identification technologies, cybersecurity 
are affecting human thinking, behavior and habits in their 
daily lives. 

Understanding human thinking is crucial if we want to, create 
technologies, which correspond and satisfies the human 
needs. It is not only question of passive adaptation of human 
beings; it is question of human beings accommodation and 
assimilation in modern information society. 


How can human behavior and thinking models be more 
deeply understood? We should study more detailed: a) human 
being primitive behavioral models (instincts, autonomous 
behavior, habits), b) deep emotions (falling in love, fear, 
betray, abuse), c) collect empirical research data from 
different user groups, artifacts, environments and task 
settings (train, metro, airport, hospital, home, work place, 
shop etc.) d) investigate organizational behavior and e) 
create new research methods (creative art methods). 
Nevertheless, it needs to be remembered that due to cultural 
differences,  humans' backgrounds, perspectives and 
motivations, humans' interpretation of the wicked problem 
varies greatly case by case. (Pavie & earthy 2014, 5). 
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The rational and emotional minds are not separate units. 
Therefore, we also need to study instincts and deep emotions 
of human being.The rational cognitive thinking models cannot 
purely explain by human behavioral and habits. We need to 
be able to better understand human being primitive behavioral 
models (instincts). The primitive behavioral models are not 
distinguished. For instance intuitiveness could be explained 
more detailed if we could understand what part of our 
behavioral is based on instincts. The Card et al. [4] used the 
concept information scent in this context. As animals rely on 
scents to indicate the chances of finding prey in current area 
and guide them to other promising patches, so do humans 
rely on various cues in the information environment to get 
similar answers. Human users estimate how much useful 
information they are likely to get on a given path, and after 
seeking information compare the actual outcome with their 
predictions. When the information scent stops getting 
stronger (i.e., when human no longer expect to find useful 
additional information), the humans move to a different 
information source. Maybe investigating animal behavioral 
model we could understand the human primitive behavioral 
models and we could create more convenient products and 
services, which respond our needs. 

At the moment, the methodical development of human 
thought and action research has focused in neuroscience. 
Beside of that we need to create new research methods. The 
creative art and agile design thinking methods has not been 
implied to rational cognitive research settings before. 

At the end, research problems are becoming more complex 
and more holistic view of different research disciplines are 
needed. The technological, neuropsychological, HCl-research 
and cognitive communities of sciences presents different 
models of cognitive thinking. Similarities and differences has 
been identified in this paper. 

The futurologists has focused to extrapolate present political, 
economic, society and environmental trends on attempting to 
predict future trends. During the recent years, the discipline 
has put more and more focus on the examination of social 
systems and wicked problems to be able to draw the future 
scenarios. Maybe futurologist could also help cognitive 
Science to predict and form more holistic picture of 
human thought and action. 
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Abstract — A new compact Specim IQ hyperspectral camera working in the 400-1000 nm 
range has been launched on the market. Its use in the investigation of different artworks and 
under diverse environmental conditions will be presented. 


INTRODUCTION 


Presently, the study of the materials constituting artworks and archaeological objects can 
be performed using invasive and/or non-invasive approaches [1]. Preliminary non-invasive 
approaches are always necessary at the beginning of the study and/or investigation of a work 
of art, or before initiating any conservation procedure on valuable objects, in order to assist 
curators and conservators in their decision-making process. Non-invasive methodologies are 
mainly performed by investigating small (spot size or 1-D technique) and wide (2-D 
technique) areas, and are focused on the diagnosis, study and conservation of, as well as on 
the access to, objects of art. Among the imaging techniques a very important role is played by 
Infrared Reflectography (IRR) and Vis and NIR hyper-spectral imaging (HSI) techniques [2- 
5]. The former enables the visualization of details underneath the visible surface of a painting, 
exploiting the partial transparency of the painted layer when subjected to IR radiation. HSI, in 
addition to the IRR information, provides a spectral and colorimetric characterization of the 
entire painted surface. 

For more than a decade, a number of studies and research projects have been devoted to 
customize hyperspectral imaging techniques to the specific needs of conservation and 
applications in museum context [5]. In view of that, HSI has nowadays reached the stage of 
mature technology and is ready to its large-scale applications. Hence, a novel concept of 
hyperspectral camera - featuring compactness, lightness and good usability - has been 
developed by Specim, Spectral Imaging Ltd. (Oulu, Finland), a company in manufacturing 
products for hyperspectral imaging. The camera, model Specim IQ, is proposed as new tool 
for novel applications in the field of Cultural Heritage. The novelty of this device relies in its 
reduced dimensions and weight and in its user-friendly interface, which make this camera 
handier and reasonably priced than conventional hyperspectral instrumentation. The camera 
operates in the 400-1000nm spectral range and can be mounted on a tripod. It can operate 
from short-distance (tens of cm) to long distances (tens of meters) with different spatial 
resolutions [6]. These technical features furtherly extend the possibility of applications of HSI 
to new typologies of artworks, both indoor and outdoor, from large-size wall paintings, 
ceilings, decorative elements and inscriptions to façade of historical buildings and 
monuments, as well as inspection small details of interest selected on the surface of almost 
flat objects. 

The imaging system, presented on November 30" 2017 in Helsinki, has been tested before 
its forthcoming official on the market by IFAC-CNR Applied Spectroscopy research group 
laboratories. In the present communication, some of the first applications to different artworks 
and under diverse environmental conditions of one of the Specim IQ portable camera 
prototypes are reported. 
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Specim IQ camera: technical information 


The Specim IQ has been designed to provide a full hyperspectral tool with required 
features for making hyperspectral imaging possible on different kind of environments (Fig. 1). 
It differs from other hyperspectral cameras on the market by integrating the hyperspectral 
sensor with additional color cameras, replaceable data storage and batteries, data acquisition 
and processing electronics, and an optimized operating system and user interface into a single 
portable housing. The integrated color cameras support the spectral camera usage by making 
it possible to direct and point the camera with standard viewfinder image as well as adjust the 
manual focus of the spectral camera with a normal camera image. By removing the need for 
additional computers, cabling, power supplies and software, it enables the users to exploit this 
novel technology in an easy and straightforward way. The rechargeable batteries and standard 
32 GB SD memory card for the image data allow approximately 100 measurements to be 
made with camera without the need to recharge or change the storage media. The operating 
system is designed in a way that it will guide the user to consider the necessary camera 
adjustment and data quality validations, without a need to go into details of the hyperspectral 
imaging technology details. The target on the system design has been to enable users, not 
familiar with hyperspectral imaging, to successfully start using it in their applications. 


Fig.1 - Specim IQ hyperspectral camera 


The hyperspectral data acquisition with Specim IQ can be made both outdoor and indoor 
conditions, with Sun light or artificial, broadband illumination. Depending on the application 
field, both LED and halogen-based illuminations can be used. In addition to the Specim IQ 
and possible illumination, a reflectance reference targets are needed to correct the effect of the 
illumination and ambient environment effect from the data, and make the measurements made 
in different conditions comparable with each other’s. Like majority of the hyperspectral 
cameras, Specim IQ does not provide the image with full spectral content with single 
snapshot, but the hyperspectral image is acquired by making a line-scan over the target area. 
The camera is designed in a way, that there is no need to move the camera or target, but the 
image scanning is performed with internal mechanisms. Due to image collection by scanning, 
the process takes in normal conditions from seconds upwards, so it is recommended to use 
Specim IQ with a standard tripod. 

The camera visualizes the hyperspectral data immediately after the measurements and the 
user has possibility to add metadata to the measurement. It is also possible to use the bundled 
Specim IQ Studio software to create material identification models for the hyperspectral data. 
These can be installed as applications to the Specim IQ, and when operating the camera with 
an application, the camera will also process the data and provide the processing results 
visualization for the user. In addition to the bundled software, the hyperspectral image data 
format is compatible with majority of the other hyperspectral data processing software’s 
available in the market. 

The Specim IQ covers the 400 — 1000 nm spectral range and provides 7 nm spectral 
resolution with 3.5 nm spectral sampling — suitable for majority of the materials having 
spectral response in this wavelength range. The resulting image from Specim IQ is 512 x 512 
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pixels, with all the pixels containing 204 spectral samples. The camera saves both 
unprocessed and processed data and the single measurement is approximately 300 MB (Table 


D. 


Table 1 Technical specifications of the Specim IQ hyperspectral camera 


Wavelength band 400 — 1000 nm 

Spectral resolution FWHM 7 nm 

Spectral bands 204 

Image resolution 512 x 512 pixel 

F/number 1.7 

Peak SNR > 500:1 

Data recording time ~ 1 s — 260 s (Depending on the illumination level) 
Object distance 150 - oo mm 

FOV 40 deg 

FOV at 1 m 0.5x 0.5 m 

Camera interface USB Type-C 

Data format SPECIM Dataset with ENVI compatible data files 
Size 207 x 91 x 126 mm 

Weight 1.3 kg 

Operational temperature +0°C - +40°C 

Operational humidity < 95% non-condensing 


Specim IQ camera: case studies 


The first case-study was performed at the Stefano Scarpelli painting conservation studio on 
the 19" century canvas painting “Saint Catherine carried by the Angels on Mount Sinai” 
(Santa Caterina trasportata dagli Angeli sul monte Sinai, 49.6 cm x 67 cm, private collection) 
by the Austrian painter Karl von Blaas (1815 - 1894). 


| i h 
Fig. 2 VNIR Specim IQ camera during the acquisition of the hyperspectral imaging data. 


The data acquired were used to identify some of the pigments used by the artist and to map 
their distribution over the region of interest by using the spectral angle mapper (SAM) 


71 


procedure that is incorporated in the software uploaded in the camera itself (Fig. 2). It was 
found, for instance, that the pigment used for depicting the blue and bluish areas was 
Thénard’s blue (also known as cobalt blue) pigment, an important synthetic pigment used by 
artists since the 19" century. In Figure 3 is reported the image that appeared in the screen of 
the camera in which it is reported on the blue right sleeve of the angel the pixel from which 
the reflectance spectrum, reported in the graph at the right, was extracted. The absorption 
features in the 500-700 nm range and the whole spectral shape of this spectrum made it 
possible to identify the blue pigment as cobalt blue [7]. 


| Fig. 3 image displayed in the Specim IQ back screen with reported the reflectance 
spectrum of the selected pixel. 


Afterwards, the areal distribution of this pigment was mapped using the SAM procedure 
with a wide tolerance thresholding option selected from the set maximum angle (in radians) 
area by using the SAM mask tool, as reported in figure 4. 


Fig. 4 distribution map of the pigment cobalt blue (in white) on the imaged area of the 
painting using the SAM procedure of the software of the camera and as reference the 
spectrum selected in figure 3. 
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Subsequently, it was tested the usability of the Specim IQ camera for remote acquisitions 
on outdoor large size painted surfaces, hardly to reach because of their location, such as one 
of the painted lunette in the Sant’ Antonino cloister of the San Marco Museum in Florence [6]. 
This cloister has been decorated with mural paintings by different artists over three centuries 
starting from the 15° century. In the 17" century, a cycle of lunettes depicting scenes from the 
life of Sant'Antonino, who was responsible for the rebirth of the monastery and an important 
figure for the Dominicans monks. The investigated lunette is located in the upper part of the 
north wall of the cloister and was painted by the Florentine painter Fabrizio Boschi (1572- 
1642)(Fig. 5). 


EM 


Fig. 5 VNIR hyperspectral camera during measurements in the Sant’ Antonino cloister at 
the Museum of San Marco, Florence. 


Finally, the imaging system was tested on a 15" century Florentine illuminated book of the 
Biblioteca Medicea Laurenziana in Florence [8]. Some of the miniatures of the book have 
been attributed to Beato Angelico, although the manuscript attribution is still subject of 
investigations. 

The data acquired on the easel- and wall-paintings as well as the illuminated manuscript 
evidenced the noteworthy potentialities of Specim IQ camera in the Cultural Heritage 
application field. Moreover, its use makes it possible to extend the use of hyperspectral 
imaging technique to new categories of artworks or artworks /archeological objects located in 
unfavorable environments, which result difficult to be investigated with the most traditional 
models of hyperspectral devices. 


Summary 


From the first data acquired with the first release of the Specim IQ compact hyperspectral 
camera it was found that this camera is a perfect imaging device for preliminary, quick 
diagnostics on different typologies of 2D polychrome artworks and in different environments. 
The performances of the Specim IQ camera in terms of spatial resolution and spectral interval 
coverage indicate that this device cannot be considered as a substitute of high-precision 
hyperspectral instrumentation specifically tailored for applications on paintings, especially 
when high-quality image documentation is required. However, it can be considered as an 
ideal tool to complement other analytical techniques and to guide the decision making process 
in planning diagnostics campaign which encompass a multi-analytical approach. 
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ABSTRACT 

One of the greatest masterpieces of Flemish art of global importance has been analyzed, through 
geological and geomorphological evidence, in order to identify the ‘hidden landscapes” depicted within 
it. The conclusions of this study suggest that the views may be Sicilian. This fact is particularly 
important as it is believed that the masterpiece was imported into Sicily, but the history of its origin is 
not yet well known, indeed, it is still under discussion. Reasonably, therefore, in the light of the results 
achieved, the possibility that it has been realized on the island itself can be seen. 


INTRODUCTION 

A manuscript of the XVIII century defines the painting of «cossì bella sottile artificiosa, et 
meravigliosa fattura, che par non da mani humane ma angeliche depinto» («so beautiful subtle 
studied, and astonishing workmanship, that was painted by angelic hands, not human» [1]. It is 
undoubtedly the masterpiece of a Flemish artist. At the beginning the painter, active in the XV 
century, was considered anonymous. From Friedlánder [2] on, he was defined as ‘Meister mit dem 
gesticketen laub”, ‘Maitre au feuillage en brodeire”, that is ‘Master of the embellished foliage”. The 
traditional assignments of the work — except for the most ancient and remote to Diirer or Van Eych 
— have always swung among Memling, Hugo Van der Goes and, more unanimously, Rogier Van der 
Weyden, pseudonym of Rogier de la Pasture [3,4,5,6,7]. 

In the central panel the Virgin with Baby Jesus on her knees is depicted, she is seated on the 
throne under a sort of fabric baldachin. There are four angels next to her. In the side panels Saint 
Katherine of Alexandria and Saint Barbara are illustrated seated and in meditative attitude. Both of 
them carry the attribute of their martyrdom (fig.1). 

Recently the Triptych has been at the centre of a futuristic project, the “l’Iperion Ch.it”, through 
which it has been possible to analyse the work with cutting-edge technologies, to better guarantee 
the renovation and preservation. In the same project there are also involved the mosaics of Pompei, 
the paintings by Pollok at the Guggenheim in Venice and the altarpiece of Saint Bernardino by 
Piero della Francesca, kept in Milan. 

The story of this masterpiece is covered by mystery. As far as seventeenth-eighteenth-century 
sources show, in the wake of a lost manuscript, it has been handed down its unlucky arrival in 
Polizzi. Indeed the precious work was delivered from Luca Giardino, captain of a ship survived to a 
storm, to a monk that he met when he arrived in Palermo. The monk immediately brought it to 
Polizzi through a series of other exciting vicissitudes [8]. 

Apart from the really surprising dimensions (162x237 cm), the masterpiece confirms that at the 
beginning of the sixteenth century even in Sicily and in the Madonie there was a predilection for the 
Flemish painting from noble environments, court and ecclesiastics. 

The execution of the work, oil on durmast wood, is very accurate and painted in great detail. 
Everything is made with mastery: the embroidery of the dresses, the carvings of the throne, the 
nestled gems, the flora, even the scroll with the musical notation that refers to the well-known Ave 
Regina by the English composer Walter Frye [9]. 
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METHODS AND RESULTS 

In the study the new discipline of the landscape busting has been applied. It represents the avant- 
garde in culture, connecting the geology-geomorphology with the human-artistic disciplines in a 
relation of synergy. This scientific-cultural original cross method can pave the way to surprising 
results in research, scientific divulgation and fruition en plein air of the artworks. 

This approach was first studied by the researchers who traced back the “invisible landscapes” by 
Piero della Francesca in Montefeltro between Marche and Romagna [10]. These places are the 
backdrop of the well-known “Gioconda” (Monnalisa) by Leonardo da Vinci in the Valmarecchia 
[11]. The same approach was also found in the landscape of the famous “Crocifissione” by 
Antonello da Messina kept in Anversa, which has been identified into the Stretto di Messina [12]. 

The method used to identify the landscape in the painting by Rogier van der Weyden is based on 
the observation of both the pictorial work and the real view. The technical analysis has been 
followed by a bibliographic research that has allowed the finding of a particular perfectly dated 
cartography. Its study has permitted to verify and compare the landscape of that time with the 
elements of the area recognised in the painting. 

In this very detailed representation of the Triptych of the ‘Gran Signora’ (big Lady), there is also 
a small but peculiar landscape detail, which is painted in the central altarpiece, behind the two 
angels that play the flute and the lute. It is a characteristic relief that, after a careful observation, can 
be identified with “Monte San Calogero”, geologically part of the mountains of Termini Imerese. It 
is the background of Polizzi in North-West direction. The comparison between this detail 
represented in the painting and the picture taken from the central square of the village shows that 
the two profiles match each other from a morphological point of view (fig. 2). 

Indeed, undoubtedly both reliefs raise themselves isolated from a valley and present a truncated 
conic form with an indented summit (white dotted arrows). In particular, on the right side for the 
observer, both of them exhibit a small tooth (white dotted arrow). The similarity is more evident if 
the photo is compressed. The compression was already known with Piero della Francesca and 
Leonardo da Vinci and it was a technique that allowed to capture an extended landscape in a 
painting. Leonardo often used it in a complex way to paint the “Gioconda” (Mona Lisa). Even the 
bluish color of this detail suggests the lithology of the impressive massif, formed of limestone and 
dolomite originated in the Mesozoic. Under the geological point of view “Monte San Calogero” is a 
big anticline, broken by normal faults, arising towards North with relevant displacement [13]. One 
of them is well put in evidence in the painting (red dotted line). 

Moreover, two skylines can be noticed faraway. The first is a tabular upland (yellow dotted line), 
which seems leant on the right side of the mountain and probably represents “Monte Pellegrino” 
with the extremity of “Monte Gallo”. Both belong to the so-called “Monti di Palermo” (mountains 
of Palermo). The second, imperceptible, wears away towards the left (yellow dotted line): it is 
“Monte Catalfano”, which dominates the city of Bagheria. 

The landscape continues in the right side of the painting, where there are other details that can 
further confirm its real location. Behind Saint Barbara, seated and covered by a dark red mantle 
with a palm in her hand, it can be noticed the architectural perspective belonging to the Flemish 
world and to the descriptivism typical of the sumptuous painting of the Netherlands of the second 
fifteenth century. The Flemish city with its noble palaces, towers, entrance door, lies on a series of 
morpho-stuctures, which look like some little hills. It can seem a chance, but this detail is shown a 
few years later (1548) in another pen and watercolour drawing. It is kept in Rome in the “Angelica” 
Library and it stylises the city of Polizzi Generosa [14]. On the right side, indeed, it can be seen the 
castle with its towers, the walls and the entrance door to the city: coincidentally, everything lies on a 
series of small mountains (fig. 3). 

Looking for documentary proofs from the past, it has been very helpful the observation of 
paintings by minor painters who worked in Polizzi Generosa. Among these there was Joan Matta, 
the ‘nobilis’ Mata, de Matta, Mat, Matt, Matth. As recent documents attest, in the first twenty years 
of the sixteenth century he moved from the Reign of Valencia to Polizzi. He was surely attracted by 
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the booming economic conditions of the city, that guaranteed him private and public orders for long 
time [15]. The mentioned work is the “Compianto sul Cristo morto con i santi Sebastiano e Caterina 
d'Alessandria” (that is the “Grief on the dead Christ with the Saints Sebastiano and Caterina of 
Alessandria"), at the moment kept in the Church of Saint Girolamo (fig. 4). It was realised around 
the third decade of the XVI century. At the top, on the right, it shows a small mountain made bluish 
by the truncated conic shape. From there, two jagged mountains raise, probably separated by a line 
of fault. Could it be a further reference to “Monte San Calogero”? We like to think it is. 


DISCUSSION AND CONCLUSIONS 

After these assertions, it is obvious to wonder some questions. Is it possible that the Flemish 
author of the Triptych knew Polizzi Generosa? Could the painting have been realised in loco? Is it 
possible to track the name of the author among the Flemish artists who travelled in Sicily? These 
are unavoidable questions and to find an answer is not easy. We can only make hypothesis, even 
though they are not proved by evident proofs. On the other hand, the whole history of the Triptych 
is mysterious, from the name of its author to its arrival in Sicily. However, this does not alter its 
importance. Certainly, the art of the fifteenth century found out the ‘real’ in its wider meaning. In 
most of the north countries a religious sensitivity spread out: it looked for a closer relationship 
between God and the human being, such that it encouraged the identification with the divinity. This 
different spirituality was one of the reasons that pushed the artists towards a figurative more 
realistic research, which paid attention to the details of the daily life. The backgrounds of the 
Renaissance works were not pure invention or result of the big evocative power of the artists. They 
were, instead, fascinating real landscapes, known by everyone. Therefore, analogously to the Italian 
contemporary painters, also the Flemish ones showed an interest more and more marked for the 
reality and the naturalistic representation. 

In addition, the characteristic and happy geographic position of the main Flemish cities 
stimulated the trade with the other European cities. As a result, there was an increase of the business 
travels, along with the study, work and pleasure ones. Merchants, bankers, diplomats, cartographers, 
scientists, masters in every art began to move from a place to another [16]. 

To remain in the context of our analysis, many historical documents, for instance, tell that Van 
Eyck travelled a lot on behalf of the duke Filippo III. It is known the one in 1428-1429 in Portugal 
to portray Isabella, Filippo’s fiancée, but also other diplomatic expeditions. In addition, in his 
paintings there are naturalistic elements that show his knowledge of the landscape and of the typical 
colours of the Mediterranean world. The representation of the bluish mountains in the background 
of some of his well known paintings let think about a possible trip in Italy [17]. 

It is then legitimate to believe that other famous names, like Van Eyck, left the Flanders to 
explore different places, following caravans that moved for business reasons or because called by 
the Royal European Houses or simply for craving for knowledge. 

Hence, it is very likely that famous painters went to the inner parts of Sicily and here accepted 
commissions and left works of great perfection. The backgrounds were very accurate landscapes 
connected to emotions so intense to be represented in a painting, as a memory of that sweet upset 
which went through the soul and arrived to the mastery of the painter. Such hypothesis could be 
related to the Triptych of Polizzi Generosa, with its “Monte San Calogero” at the horizon. It could 
have been painted in loco by a Flemish artist arrived in the island during one of his trips. Or it might 
have been sketched on a notebook and then realised in homeland, going through memories and 
spontaneous feelings, whose intense taste could still be perceived. 
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Fig. 1. Rogier van der Weyden (attributed), Triptych of Polizzi Generosa, XV century ( 
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Technological and digital innovation is becoming more and more rapid and pervasive in its 
use and diffusion by tapping an ever-wider spectrum of processes. 

These new generation technologies are increasingly integrating themselves and 
integrating various cultural heritage areas, such as the conservation, enhancement, and 
enjoyment of cultural heritage, and find applications in various projects such as increased 
reality, the /nternet of Things and digital manufacturing. 

This leads to new forms of musealization, fruition and dissemination of artistic and cultural 
heritage, in which the technological components make the difference and give the 
cognitive and emotional experiences a very important role; consequentially it is 
characterized a guest who is increasingly expected to be the protagonist of this cultural 


experience and not a passive spectator. 


First of all, we are faced with an enhanced web, capable of modifying cultural aspects 
thanks to its content and the power of the media, from which we get a continuous 
exchange of information. 

This ongoing flow leads the user to address the digital culture on three levels of time: 

- BEFORE: "before fruition" the user is informed, having access to a news archive of any 
kind, ranging from those directly provided for example by the museum, to those of other 
users who evaluate their experience, comment and express opinions. 

For example, if we decide to visit a museum, we can get information from two 


macrochannels: directly from the official website, where we will find detailed information on 
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timetables, fares and collections details, or from reviews and opinions channels such as 
Trip Advisor or channel company. 

In this case we will have other information that we would not find on "classic" channels, 
such as what works deserve absolutely to be seen, as they liked most to users who have 


already gone to the same museum. 


- DURING: "during enjoyment", that is, all that digital set of actions that the user performs 
directly when they receive the information. The use of interactive maps, smartphone apps, 
audio guides, increased reality, Li-Fi systems; all times when the user feels involved in the 
path and experience he is facing. 

When we are in the museum or sometimes on the internet site we can immerse ourselves 
in virtual and interactive experiences that help us understand the work better and more 


deeply while guiding us along the way. 


- AFTER: "after fruition", the user re-submits contents, comments, reviews and 
experiences to the web, as if he was still enjoying the work, giving new imput to users who 
are still beginning to enjoy. 

Concluded the cultural experience we are now accustomed to "sharing", making public 
what we did, what we saw and how we felt to see it. By emulation or passion for the 
protagonist, nowadays most people use social, and even those who do not, by social 


status will be brought to tell their own experience that will be heard and shared with many. 


So doing is fueling an acceleration of the irreversible phenomenon that culture produces; 
that is, a spiral loop that is self-feeding and pushes itself further: thanks to culture and with 
culture, another culture is created. 

If the phenomenon exists for so long, today there is an implementation of speed. 

A way to produce culture different from the one practiced until recently, driven by new 
technologies and users that can not be controlled. This results in the creation of a 
spontaneous communication in which the originality and truthfulness of the information is 
not known. Just refer to the scandalous expansion of Fake News. In addition to veracity, 
the manipulation of information has largely focused on attention for speculative purposes. 
The case of the scandal that invested TripAdvisor on "bought" reviews is a significant 


indicator. 
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The risk is to get to a point where only the "macro elements" of our cultural heritage will be 
known or those who will have a "sponsor" with greater influence. In the field of fashion, for 
example, this trend has already come into play with the entry of Fashion bloggers, 
Ambassadors and Influencers, so we could easily suppose, without even a great effort, 


that this could happen in other cultural areas. 


As is well known, virtual and multimedia languages can thus have a revolutionary impact. 
Thanks to the unlimited ability to tell stories, the ability to rebuild the worlds and objects 
lost today is becoming available, plus the ability to enjoy it enthusiastically and emotionally. 
They can also support the documentation by creating digital, 2D and 3D archives that are 
useful to the entity that manages them, as well as stimulating new stimuli of interest and 
public affection for cultural goods. Digital archiving guarantees the indelible preservation of 
the work spontaneously triggering actions of care and safeguarding, knowledge and 


enhancement. 


This abundance of digital data, as it creates new scenarios of new perceptual knowledge, 
crumbles the expectations that we have been accustomed to having in the near to the not 
yet known. Everything happens before it is lived. 

Until recently, to see a work of art we had to go to a certain city, in a particular museum, 
and we did not know what was waiting for us, as it would have been to see it close, while 
now we can see it on the screen of our pc in very high resolution to the slightest detail, so 
in that sense you lose the surprise in having it really in front because somehow we have 


already seen, already experienced, we have already created an opinion about it. 


The new opportunities that cultural institutions now have, increasingly require more 
information technology to better exploit cultural heritage. 

Digital manufacturing can in fact be used in integrative restoration, digital reconstructions 
and the creation of high-definition models, in tactile alternatives for disabled and children, 
and also in museum merchandising. Through the 3D survey, it is possible to create digital 
archives in three dimensions of artefacts, artefacts and artworks for use both for 
dissemination purposes, such as the development of 3D online museums, as well as 
tourist applications and integration to cultural enjoyment. The use of this type of technology 


was initially limited in the useby high-cost and low-performing devices for the scholars, 
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while the general public has had a difficult approach due to the low availability of intuitive 
tools that they stopped using ease of use and immediate accessibility to real-time 


navigation. 


However, the evolutionary acceleration of these technologies has allowed a widespread 
ramification of the success of these IT tools, making it easier to access and approach any 
level of use whether it is for researchers and operators in the industry, whether they are 
users concerned. An evolution that allows an ever-expanding dissemination of cultural 


heritage. 


In this new perspective, Design plays a significant role. The 3D digital medium holds 
sensitive potential in terms of beneficial or disadvantageous relaunches for the 
dissemination of cultural heritage. 

If digitization allows conservation and facilitation in propagating the value of the artwork, at 
the same time the problem of ease of counterfeiting is evident. 

The copyright protection is, in fact, one of the most delicate issues for the 3D printing 
industry especially with the advent of mass 3D. 

We could make a comparison with what happened a few years ago in the music world. 
After all, a .stl file is a kind of .mp3 of the objects. With the advent of this type of file, the 
music industry has been invested by a real tsunami that has left behind its pros and cons. 
Looking at the positive, we've come to an extreme usability of music content, but have also 
had negative consequences, such as ever-expanding digital piracy. 

Such a scenario is reappearing with the advent of massive 3D printing, and it will not be 
enough to hide behind the idea that engraving a CD is much easier than printing objects in 
3D because at the time, even for CD playback, we did not even take this easy access to 


music content for granted. 


A very important and relevant issue is that of digital file protection, especially in the field of 
cultural herritage or design. 

An increasingly emerging need to avoid counterfeiting of works and models, which will be 
increasingly subject to faithful reproduction of the original. As is known about this trend, 


highly innovative solutions have been developed and are still being studied but not yet 
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sufficient to contain or avoid counterparts which are acceptable to support the contribution 
of technological emancipation to knowledge and protection of cultural goods. 

The contradiction between preserving, protecting, guarding and counterfeiting demands 
also implicitly impacts the use of digital technologies in designing new objects. 

3D modeling assists each design phase up to the real simulation with the three- 
dimensional automated printing of the object itself. This allows you to have enormous 
possibilities of formal expression through digital tools that accelerate the process of 
visualization of an idea and make it even easier for the isometric control already visible in 
the virtual field before it materializes. That's the role of digital modeling assumes a value 
that asks distinction of identity as it happens in the authenticity of a work of art. 

Both a monument and a design object need to be protected. The artwork and design 
object that can be circulated on the web, available to anyone, need more protection. 

It follows that in order to benefit from digital technologies, it is necessary to increase the 
awareness of the type of use of computer tools and how to preserve their autoriality in 


every step of the dissemination. 


Particularly with regard to Design, we can notice the deficiencies of this awareness caused 
by two major factors. 

The first as a result of the diffusion of IT technologies in the project culture that enable 3D 
modeling a more direct and facilitated approach to the preview of the product to be 
achieved. The second factor, however, concerns the subconscious subjection that 
computer tools allow those who, enticed by their performance, do not absorb their potential 
but undergo such process facilitation. 

The problem is that the tool is also predominant on the vision of the shape of things. 
These are the features, commands and tools of the program that dictate the form, not the 
design idea of the designer. In the academic sphere, this trend is very rampant, especially 
in those who, aspiring to the profession of designer, still do not know how to fully use these 
tools, which should remain the same, without becoming overwhelming on the final aspect 
of the product. 

To illustrate this trend we could make a comparison between these new tools and the 
classic designer tool, pencil. If we take a pencil and try to draw circles, we will certainly find 
more difficulty than drawing simple curved lines, but that does not mean that we will 


exclude circles from our projects; we will only have to refine the technique. 
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Today, however, we see many young designers approaching these new technologies with 
a "surrender" attitude: they do not even try to draw shapes that they know they do not 
know how to model. As a result, everything flattens into definable shapes with very few 
simple commands, such as extrusion, creating products that do not represent the research 
for a formal idea of the designer, but the degree of software usage skill with which they 


were generated. 


Gaining ingenuity, subtlety, or simple adaptability to the facilitating potential of the 
instruments, however, gives a fundamental consideration of how to deal with a future 
strategy. 

Technological innovation opens up and raises new perspectives, but at the same time only 
a very strong sensitivity to awareness of what we have and what we can do to facilitate the 
exchange between ways of acting and ways of thinking. 

A new focus that will distinguish between what we identify as a cultural heritage and what, 
through technology, makes us cultured with new skills with which we use computer 


resources. 
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Abstract 


Sub-optimal post-operative anatomical and mechanical alignment has been described 
as a cause of Total Knee Arthroplasty (TKA) revision. In fact, mechanical alignment 
exceeding + 3 has been reported to increase the risk of early failure following TKA. For this 
reason, alignment guides have been developed to improve the accuracy of femoral and tibial 
cuts during surgery. Recently, it has been introduced a new system (PERSEUS) that aim to 
help surgeons during bone cuts, reducing the complexity of conventional alignment and 
sizing tools. 

PERSEUS system is an intelligent cutting guide, powered by gyro sensors, that 
allows surgeon to perform perfectly aligned resections with less effort and morbidity, 
compared to conventional instrumentation for TKA. It enables to perform bone resections 
without using intramedullary rods, reducing the risks of a more invasive surgical approach 
and ensuring the optimal final limb alignment. With the use of a disposable sensor and a 
touchscreen interface (IPAD), the surgeon could verify the correct placement of the cutting- 
guide relating to the femur. During surgery, the real-time cutting plan can be approved or 
declined or modified, according to the choice of the surgeon. This system appears to be less 
risk and more safety 1f compared to other Computer Navigation Systems. Further, it avoids 
risks related to intramedullary nail and it is easier to use, increasing confidence in surgical 
decision. From January 2016 to March 2018, we treated 15 patients affected by knee 
osteoarthritis using the Perseus System. After post-operative CT evaluation, correct 


placement of the tibial and femoral component was obtained in all cases. 
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Abstract 


Digitization requires today high skill and great experience in order to achieve ultimate level of 
digitization results. Time is money and efficiency to get the best output qulity requires a proper 
fine-tuning set-up and proven workflow. This paper is about the experience developed by the 
Alinari Archives in Florence, Italy, on creating possibly the ultimate state-of-the-art digitization 
solution for the cultural heritage sector. Today digitization has evolved quite a lot. Thanks for the 
latest technical advances, slow scanners and digitization systems available in the mid of the 90's 
have been replaced by faster, more powerful and with higher technical specifications. It is however 
important to specify that between 1999 and 2007, the market was offering specific dedicated 
scanners for 35mm and small films, such as film scanners, drum scanners and flat bed scanners, 
capable to capture very high details and information in the highlights (ie. sky) and shadow (ie. dark 
areas like the shade of a building). Those systems had a dynamic range (capacity to capture all level 
of color/gray scale information) up to 4,8. Today technology has become better in terms of scanning 
speed, maintenance and software management. However in many cases, prices have increased 
dramatically, and most important, the market has lost high end dedicated scanners for small film 


and 35mm slides. With the market monopoly of digital imaging, starting in the mid of 2000, those 
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scanners have lost market share and since 2010 no vendors offer any high solution for those type of 
media. 

The study analysis various solutions available in recent years and also thanks to the experience 
developed by Alinari in the last 15 years around best practice on digitization, providing latest 


reccomandations related to the current solutions available in the market. 
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Abstract — FTV (Free-viewpoint Television) is visual media that transmits all ray information of 
a 3D space and enables immersive 3D viewing. The international standardization of FTV has been 
conducted in MPEG. The first phase of FTV is multiview video coding (MVC), and the second 
phase is 3D video (3DV). The third phase of FTV is MPEG-FTV, which targets revolutionized 
viewing of 3D scenes via super multiview, free navigation, and 360-degree 3D. After the success of 
exploration experiments and Call for Evidence, MPEG-FTV moved MPEG Immersive project 
(MPEG-I), where it is in charge of video part as MPEG-I Visual. MPEG-I will create standards for 
immersive audio-visual services. 


1 INTRODUCTION 


FTV (Free-viewpoint Television) [1]-[8] is visual media that transmits all ray information of 
a 3D space. FTV was developed based on ray-space representation [9]-[12]. 

FTV is the ultimate 3DTV, with an infinite number of views, and ranks at the top of visual 
media. FTV enables users to view a 3D scene by freely changing the viewpoint, as we do 
naturally in the real world. FTV is a natural interface between humans and the environment. It is 
also an immersive media that enables a realistic VR experience and revolutionizes 3D viewing. 

FTV was proposed to the Moving Picture Experts Group (MPEG) in 2001 [13]. Since then, 
the MPEG has been developing various FTV standards. Multiview video coding (MVC) [14] is 
the first phase of FTV and enables efficient coding of multiple camera views. 3D video (3DV) 
[15] is the second phase of FTV and enables viewing adaptations and display adaptations for 
multiview 3D displays. MPEG started the third phase of FTV [16] in August 2013. This is 
MPEG-FTV, which targets immersive viewing of 3D scenes via super multiview, free navigation, 
and 360-degree 3D (360 3D) video. MPEG-FTV moved to MPEG-Immersive project (MPEG-I) 
[17] in January 2017 and it has been in charge of video part as MPEG-I Visual. 

In this paper, international standardization of FTV is described. 


2 HISTORY OF FTV STANDARDIZATION IN MPEG 


The MPEG has been developing FTV standards since 2001. The history of FTV 
standardization in MPEG is shown in Fig. 1. In 2001, FTV was proposed to the MPEG, and the 
3D audio visual (3DAV) activity started. In 3DAV activity, many topics, such as omnidirectional 
video, FTV, stereoscopic video, and 3DTV with depth information, were discussed. According 
to the results of the call for comments from the industry, discussion converged on FTV and 
MVC [14] starting in March 2004. 

MVC is the first phase of FTV and targeted the coding of multiple videos. The MVC activity 
moved to the Joint Video Team (JVT) of the MPEG and International Telecommunication Union 
Telecommunication Standardization Sector (ITU-T) for further standardization processes in July 
2006. MVC was completed in March 2009. MVC does not have a function for view generation. 

The MPEG started 3DV as the second phase of FTV in April 2007. 3DV is a standard for 
multiview 3D displays [15]. View generation was introduced into 3DV to increase the number of 
views for multiview 3D displays. The 3DV activity moved to the Joint Collaborative Team 
(JCT)-3V for further standardization processes in July 2012, and 3DV was completed in June 
2016. 
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In August 2013, the MPEG started the third phase of FTV, MPEG-FTV [16], which targets 
immersive 3D viewing by enhancing the function of view generation. MPEG-FTV moved to 
MPEG-I [17] for further standardization in January 2017. 
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Fig. 1 History of FTV standardization in MPEG. 


3 FTV FIRST PHASE: MVC STANDARD 


The framework of MVC is shown in Fig. 2. MVC targets efficient coding of multiview 
video. In MVC, the number of input views is the same as output views. The view-generation 
function of FTV is not included in MVC. 

Multiview video data have a high correlation among views. This redundancy can be removed by 
interview predication. It can also be done by using a motion compensation method that is widely 
used to remove temporal redundancy in conventional video coding. MVC applies motion 
compensation-like prediction to not only time and but also view directions. 

MVC was standardized as the extension of H.264/MPEG4-AVC [18]. The MVC standard was 
adopted by Blu-ray 3D. 


multi-view 


Encoder Decoder 


Trans- 
mission 


Sender side Receiver side 
Fig. 2 Framework of MVC. 
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4 FTVSECOND PHASE: 3DV STANDARDS 


The framework of 3DV is shown in Figure 3. View synthesis was introduced into 3DV, 
which sends a small number of views and generates a large number of views at the receiver for 
multiview displays. A multiview and multi-depth set is jointly compressed and sent to the 
receiver, and intermediate views are synthesized from views with the assistance of depth 
information at the receiver. 3DV enables display adaptation and viewing adaptation [19]. 


MVD (multi-view + depth) 


multi-view P Po 
mult-depth MANA LAY. 


Fu Depth | 


Stereo Sender side Receiver side Multi-view 
camera display 


Fig. 3 Framework of 3DV. 


The FTV reference model, as shown in Fig. 4, was defined to develop the 3DV standard [20], 
and 3D warping is used for view synthesis of 3DV. View synthesis by 3D warping is sensitive to 
error in depth information. Nagoya University provided Depth Estimation Reference Software 
(DERS) [21] and View Synthesis Reference Software (VSRS) [22], as shown in Fig. 4. It also 
provided various test sequences such as pantomime, champagne_tower, kendo and balloons. 
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Fig. 4 FTV reference model and Nagoya University’s contribution. 


The data format of 3DV is Multiview plus Depth (MVD). Coding standards such as MVC+D, 
3D-AVC, MV-HEVC, and 3D-HEVC were developed [23]. Here, MVC+D is a depth-extension 
of MVC, 3D-AVC is AVC-based MVD joint coding, MV-HEVC is HEVC-based MVC, and 
3D-HEVC is HEVC-based MVD joint coding. 

Global View and Depth (GVD) [24] can be used as an alternative data format. GVD is a 
compact version of MVD and is obtained by removing the interview redundancy of MVD. 
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5 FTV THIRD PHASE: MPEG-FTV/MPEG-I 


5.1 Motivation and Background 


In 2010, the 2022 FIFA World Cup Japan Bid Committee planned to deliver the excitement 
of the soccer stadium to the world via FTV. It aimed to revolutionize the viewing of the soccer 
game by super multiview and free navigation. Super multiview realizes very realistic 3D viewing 
of the scene, and free navigation realizes a walk-through or fly-through experience of the scene. 
This became a strong motivation for the third phase of FTV. 


5.2 Framework of FTV 


Based on the above motivations and background, the framework of MPEG-FTV was created, 
as shown in Fig. 5 [25]. FTV has three types of application scenario [26]. The first is super 
multiview (SMV) with a high number of views and high density for super multiview displays. 
The second is a single view with freely changing viewpoint for free navigation (FN) in a wide 
area. Users can enjoy realistic 3D viewing and walk-through/fly-through experiences in 3D 
scenes. The third is 360 3D video with a wide FoV. 


Display 


Super-Multiview 


Capture 


A s View 
Synthesis 


Sender side Receiver side 


360 3D video 


Fig. 5 Framework of MPEG-FTV. 


5.3 Call for Evidence 


After a series of exploration experiments on FTV [27], the MPEG issued a Call for Evidence 
(CfE) on FTV [28] in June 2016. CfE is a procedure before a call for proposals to show evidence 
of a new technology that is better than currently available standards. FTV software used for the 
CfE is described in [29]. Submissions were collected for SMV and FN application scenarios, as 
shown in Figs. 6 and 7, respectively, in February 2016. Results evaluated in June 2016 showed 
clear evidence of the new technology [30]. 

FTV test material and software developed in MPEG-FTV are summarized in [31] and [32], 
respectively. 
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Fig. 6 Super Multiview (SMV) application scenario for the CfE [28]. 


Fig. 7 Free navigation (FN) application scenario for the CfE [28]. 
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5.4 MPEG-I 


MPEG-FTV moved to MPEG-I in January 2017. MPEG-I was established by integrating the 
FTV, light field, point cloud, and 360 video ad hoc groups. MPEG-I will create standards for 
immersive services. MPEG-FTV is in charge of the video aspects of MPEG-I. All application 
scenarios, requirements, test material, and software for MPEG-FTV were transferred to MPEG-I. 

MPEG-I will use various technologies, such as FTV, light field, point cloud, 360 video, and 
3D audio, to build immersive services. Therefore, the MPEG has structured MPEG-I as a suite of 
standards focusing on specific technologies. The five parts to MPEG-I are as follows [33]: 

*Part 1 — Technical Report on Immersive Media 

*Part 2 — Application Format for Omnidirectional Media 
*Part 3 — Immersive Video 

*Part 4 — Immersive Audio 

*Part 5 — Point Cloud Compression 

MPEG-I standards will be developed according to the stages of immersion shown in Fig. 8 
[33]. The stages of immersion are categorized by degrees of freedom (DoF), which denotes the 
number of independent parameters used to define movement of a viewpoint in 3D space. 
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Fig. 8 Stages of Immersion in MPEG-I [33]. 


For example, 3DoF is three rotational and unlimited movements around the X, Y, and Z axes. 
3DoF has a fixed viewpoint and no translational movements along the X, Y, and Z axes. A 
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typical use case is a user sitting in a chair looking at 3D 360 VR content on an HMD, as shown 
at the most left in Fig. 8. On the other hand, 6DoF is 3DoF with full translational movements 
along the X, Y, and Z axes. A typical use case is a user freely walking through 3D 360 VR 
content displayed on an HMD, as shown at the most right in Fig. 8. 

3DoF+, windowed 6DoF, and omnidirectional 6DoF are stages in between. 3DoF+ is 3DoF 
with additional limited translational movements along the X, Y, and Z axes. Windowed 6DoF 
denotes 6DoF with constrained rotational movements around the X and Y axes (pitch and yaw, 
respectively) and constrained translational movements along the Z axis. Omnidirectional 6DoF 
denotes 6DoF with constrained translational movements along X, Y, and Z axes. 


6 CONCLUSION 


MPEG has been creating various standards on FTV. In the first phase of FTV, MPEG 
developed the MVC standard. In the second phase of FTV, MPEG developed the MVC+D, 3D- 
AVC, MV-HEVC and 3D-HEVC standards. The current third phase of FTV is MPEG-FTV. 
MPEG-FTV targets revolutionized viewing of 3D scenes via super multiview, free navigation, 
and 360-degree 3D technologies. MPEG-FTV developed test material, reference software, and 
evaluation methods for them. After the success of the exploration experiments and Call for 
Evidence, MPEG-FTV moved to MPEG-I and has been in charge of its video part. MPEG-I will 
create standards for immersive services based on the stages of immersion. 
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Abstract —Ultra-wide area FTV is an FTV with very wide viewing zone where motion parallax is 
realized. 4D orthogonal ray-space is analyzed and applied to ultra-wide area FTV. Ray-space of “a 
group of rays through one point” is derived in 4D orthogonal ray-space. It is extended to obtain 
ray-space captured by linear arrangement cameras. View generation of ultra-wide area FTV needs 
rays that are not captured by real cameras. These rays are synthesized by interpolating the 
captured ray-space so that the intersections of the captured ray-space and the ray-space of rays 
emitted from a light source have the same color. 


1 INTRODUCTION 


FTV (Free-viewpoint Television) [1]-[8] is visual media that transmits all ray information of 
a 3D space and enables immersive 3D viewing as if we were actually there. Omnidirectional 
FTV is an FTV with very wide field of view (FOV), that is 360-degree FOV. Ultra-wide area 
FTV is an FTV with very wide viewing zone that gives motion parallax. FTV was developed 
based on ray-space representation [9]-[12]. 4D ray-space is needed for immersive 3D viewing 
with full parallax. 

We proposed two types of ray-space, orthogonal ray-space and spherical ray-space. 
Orthogonal ray-space is used for linear and planar camera arrangements and spherical ray-space 
is used for circular and spherical camera arrangements. 4D spherical ray-space was analyzed and 
applied to omnidirectional FTV [13]. In this paper, 4D orthogonal ray-space is analyzed and 
applied to ultra-wide area FTV. Ray capture and view generation in 4D orthogonal ray-space are 
presented. 


2 4D ORTHOGONAL RAY-SPACE 


2.1 4D Orthogonal Ray-Space of Rays through One Point 


Fig. 1 shows the definition of orthogonal ray-space. As shown in Fig. 1, one ray is expressed 
by intersection (x, y) and direction (0, o) on a reference plane. Therefore, one ray is expressed by 
one point in 4D parameter space (x, y, 0, g). This parameter space is 4D orthogonal ray-space. 
Each point in the 4D ray-space has an intensity f of the ray. Therefore, fis expressed as f(x, y, 0, 
9). 

Here, “a group of rays through one point” is considered as shown in Fig. 2. This concept is 
used for ray capture, view generation and ray synthesis as shown in Fig. 3. Fig. 3(a) shows ray 
capture by a real camera, and view generation by a virtual camera collecting rays through one 
point. Fig. 3(b) shows ray synthesis. If the surface of an object is a diffusion surface, one point 
on the surface emits rays with equal magnitude. Therefore, if one ray is known, other rays can be 
synthesized. 
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Fig. 2 Concept of “a group of rays through one point”. 
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(a) ray capture/view generation (b) ray synthesis 

Fig. 3 Ray capture, view generation and ray synthesis by using the concept of “a group of rays 
through one point”. 


x and y of a ray with direction (0, p) through a point P(X, Y, Z) are given by 


x=X — tan0Z (1) 
y lang 
ps cosÜ a (2) 


(1) and (2) give the 4D orthogonal ray-space (x, y, 0, p) of a group of rays through one point. By 
changing 0 and o, this group of rays forms a plane in (x, y, 0, 9) space. To visualize it, (x, y, 0) 
space with fixed g and (x, y, p) space with fixed 0 are considered in the following. 
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From (1) and (2), the following equations are derived. 


1+(5 3! 6) 
1+ (tan)? = ge) (4) 


For fixed o, (1), (3) and (4) give the top view, front view and side view of (x, y, 0) space, 
respectively. The top view is a straight line with slope -Z and both the front and side views are 
hyperbolic curves in (x, y, tan0) space. Therefore, a group of rays through one point forms a 
hyperbolic curve in (x, y, tan0) space with fixed g as shown in Fig. 4. Three examples of 


hyperbolic curves for o 0, p=0 and o « 0 are shown. 
For fixed 0, (1) and (2) form a straight line in (x, y, tano) space as shown in Fig. 5. Three 


examples of straight lines for 07 0, 0=0 and 0«: 0 are shown. 


z > 
@ fixed 


Fig. 5 Rays through a point P(X, Y, Z) form a straight line in (x, y, tang) space with fixed 0. 
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2.2 4D Orthogonal Ray-Space Captured by Linear Arrangement Cameras 


Let arrange many cameras on a line that is set on the xz-plane (y=0) and parallel to the x-axis. 
When an omnidirectional camera at P(X., 0, Z.) on the line captures a ray with direction (0, 9) as 
shown in Fig. 6, x and y are given by 


x= X, — tan0Z, (5) 


y= = tanp=Z.tang t + tan°0 (6) 


(5) and (6) give ray-space captured by linear arrangement cameras. 


< 


position 
SR plane y a ! (x, y) 
Ñ diréction 
(8, 9) 


X 


Camera 
(X, 0, Z) 


Camera line 
(v=0, z-Z, Z.>0) 


N 


Fig. 6 Ray capture by linear arrangement cameras 


By varying X., (5) and (6) form a hyperbolic cylinder in (x, y, tan@) space with fixed 9 as 
shown in Fig. 7. They form a plane in (x, y, tang) space with fixed 0 as shown in Fig. 8. The 
expressions of the hyperbolic cylinder and the plane are given by 


y=Z.tang, /1 + tan°0 foranyx (7) 


and 


Zo 


così 


y= tang for any x (8) 


respectively. 
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Fig. 7 Ray-space in (x, y, tan0) space with fixed y captured by linear arrangement cameras. 
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Fig. 8 Ray-space in (x, y, tang) space with fixed 0 captured by linear arrangement cameras. 


3 VIEW SYNTHESIS FOR ULTRA-WIDE AREA FTV 


4D orthogonal ray-space is applied to ultra-wide area FTV [11]. Ray capture and synthesis 
for ultra-wide area FTV is shown in Fig. 7. Rays are captured by many cameras on a line (y=0, 
z=Z,) and virtual camera views are synthesized from the captured rays. Let one of the rays 
captured by a virtual camera be (xo, vo, do, go). This ray is synthesized because it is not captured 
by real cameras. When z of the light source of this ray is assumed to be Z, the position (X, Y, Z) 
of the light source is determined. This light source emits rays in various directions. Let ray-space 
of the emitted rays be (x, y, 0, y). 
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72) light source is varied 
along ray D by 
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camera line (y=0, z=Z,) 


D real cameras 


ray to be synthesized 
(Xo Vo do Po) 
Fig. 9 Ray capture and synthesis for ultra-wide area FTV. 


Some of emitted rays are captured by the real cameras. These rays are given by the 
intersection of emitted and captured ray-spaces. Fig 10 shows captured ray-space, emitted ray- 
space and their intersection in (x, y, p) space with fixed 0. The intersection is given by 


(x, y, tang) = (X - Ztan0, 5, z (7) 


Many intersections are obtained by varying 0. Fig. 11 shows the trajectory of this intersection. 
i i LY á $ è 
It is on a horizontal plane of y= 77. When Z is correct depth, the intersections have the same 


color. This color is given to the ray (xo, yo, 0o, Po). 


Captured ray}space 


Fig. 10 Captured ray-space, emitted ray-space and their intersection in (x, y, p) space with fixed 
9. 
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Fig. 11 Trajectory of the intersection of captured ray-space and emitted ray-space in (x, y, 9) 
space by varying 0. 


The process of this ray synthesis is summarized in the flowchart of Fig. 12. 


Ray (xo. Volo Po) 


Position of light source 


Ray-space of emitted rays from light source 


Intersections of captured and emitted ray-spaces 
Variance V of color of intersections 


Update of Z 


Color of ray (xo, Vo Oo Po) 


Fig. 12 Flowchart of ray synthesis. 
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4 CONCLUSION 


Ultra-wide area FTV with full parallax is realized by 4D orthogonal ray-space processing. 
Ray-space of “a group of rays through one point” is derived in 4D orthogonal ray-space. It is 
used for ray capture, view generation and ray synthesis. Ray-space of “a group of rays through 
one point" forms a hyperbolic curve in (x, y, tan0) space with fixed g and a straight line in (x, y, 
tang) space with fixed 0. By varying the camera position along a line, ray-space captured by 
linear arrangement cameras is obtained. This ray-space forms a hyperbolic cylinder in (x, y, tan) 
space with fixed g and a plane in (x, y, tang) space with fixed 0. 

Rays for view generation of ultra-wide area FTV are synthesized by interpolating the 
captured ray-space using the concept of “a group of rays through one point”. It is done so that the 
intersections of the captured ray-space and the ray-space of rays emitted from a light source have 
the same color. 
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Cyber Ecosystem always more complex 


Digitization become deeply embedded in each strategy, as nearly all businesses and activities have 
been slated for digital transformations. 


The technological revolution has the intensity of a tsunami: the traditional ICT systems move to 
complex digital ecosystems. Operational Technology (OT) such as industrial control systems and 
elements of smart grids such as smart meters, vehicles, and smart buildings are all examples of 
innovative enterprise. 


Several new assumptions have to be made about the cyber environments, because of their evolution 
over the years: 


* Modern networks are very large, very interconnected, and run both ubiquitous protocols and 
proprietary protocols. Therefore, they are often open to access, and a potential attacker can 
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attach with relative ease, or remotely access, to such networks. Widespread IP internetworking 
increases the probability that more attacks will be carried out over large, heavily 
interconnected networks. 


+ Computing systems and applications attached to these networks are becoming increasingly 
complex. In terms of security, it becomes more difficult to analyze, secure, and properly test 
the security of the computer systems and applications; it is even more so when virtualization is 
involved. When these systems and their applications are integrated in large networks, the risk 
to computing dramatically increases. 


Warning: «Cyber space» interacts with «Reality» 


Smart devices, machine-to-machine (M2M) communications and cloud-based services, among many 
other technologies, are advancing the next-generation of networked societies. 


Digital technology and internet connectivity, are being systematically integrated into all verticals of the 
private and public sectors, offering significant advantages like productivity, speed, cost-reduction and 
flexibility. 


In the cyber environment it's increasing the role ofthe Cyber-Physical Systems (CPS), whic refer to the 
seamless integration of computation and networking with physical processes, possibly with humans in 
the loop. 


In these systems, embedded computers and networks monitor (through sensors) and control (through 
actuators) the physical processes, usually with feedback loops where physical processes affect 
computations and vice versa. 


A key point in these systems is the control of physical processes from the monitoring of variables and 
the use of computational intelligence to obtain a deep knowledge of the monitored environment, thus 
providing timely and more accurate decisions and actions. 


The growing interconnection of physical and virtual worlds, and the development of increasingly 
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sophisticated intelligence techniques, has opened the door to the next generation of CPS, referred to as 
“smart cyber-physical systems” (sCPS). 


A new family of risk factor arises, however, from the possible short-circuit between the cyber space 
(and its cyber-threats) and the controlled real world, cutting off altogether people's intervention. 


Trust challenges for the success of digital projects 


Establishing and maintaining a secure computing environment is obviously mandatory, but it's 
increasingly more difficult as networks become increasingly interconnected and data flows ever more 
freely. 


In such scenario, one methodological approach calibrated and not generalist became unavoidable. 


Security ... Common issue! 
Cybersecurity became an essential requirement when living in a digital world. 


The number of data breaches and the level of cyber-attacks suffered by organizations around the 
world increases almost daily, making it imperative that every organization have an effective 
cybersecurity program in place. 


| Business&Services Level (public, LE, SME) 
| Critical + ^ 

| -Infrastr/OoES "Industry 4.0" ¿ppp 

| Security 


Z, | OT/ICS “IoT” “CPS” Man geme. t 


"sensing, 


:onji y 
| integrity 10] data, 
Availability (of data+serviceskcapabilities) 


The develop of one cyber&information security management strategy and the relative governance 
actions are now key topics for to ensure data protection, asset defence, system resilience and overall 
continuity and capabilities of business processes and services. 
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How to cope the digital risk in the digital era 


As the global digital revolution takes hold and proliferates into our economies, societies and 
governments, the potential for information to be electronically tampered with and controlled is real. 


Digital and technology risk is a term encompassing all digital enablements that improve risk 
effectiveness and efficiency—especially process automation, decision automation, and digitized 
monitoring and early warning. 


In particular, small companies quickly moving to digitalization, often unwittingly open the door to 
attackers through a variety of unsecure practices. Small enterprise security policies that don't quell 
missteps such as employee downloads of unauthorized software, rogue Wi-Fi installations, and 
password sharing will actually promote such behaviors. 


Therefore, already tested models of reference are necessary, but unfortunately these have very 
difficult management characteristics and heavy effort for the SME target. 


To overcome the Adoption/Execution Gaps 


The needs of protection for the large or small networks are essentially the same, but the framework 
for action must to be adjusted, to response to very different operational conditions. 


For to determine the risk factors, the step is to perform a risk assessment on the organization's “cyber 
presence” in the ecosystem, by looking at information assets, interdependencies with other 
organizations, threats (including insider threats), vulnerabilities, cybersecurity controls, and security 
testing activities, including business continuity /disaster recovery and reconstruction capabilities. 


This means examining those factors that affect the extent of the organization's control over its 
ecosystem and the means by which that control can be exercised. 


* 


Oner 


Purpose: Se 


Gap radliillop 


- The solution of using more famous structured models requires an unsustainable effort for 
small but very digitized organizations (for example, robotised SME), creating a very big “gap” for 
apractical application. 
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- Moreover, some good practices that can give an added-value to new digital projects, require 
intervention by the market players who operate upstream of the deployment and are therefore 
not governable by the customer. Many of the vulnerabilities in loT/CPS devices could be 
mitigated through recognized security best practices, but too many products today do not 
incorporate even basic security measures; other contributing factors include a lack of incentives 
for developers to adequately secure products, since they do not necessarily bear the costs of 
failing to do so, and uneven awareness of how to evaluate the security features of competing 
options. 


Best practices too much «expensive» 


The most famous best practices and the most common standards (including certifiable schemes) need 
staff and resources almost always absent in small organizations. 


(as Center for internet Security N ST 


National Institute of 
Standards and Technology 


CIS-20 NIST ISO/IEC 
| CSF 2014 27000 


High effort: +Security Staff ... -IT Staff ... 
* Risk Management Processes ... +Audit Capability 


The perspectives ... 
- Quick-win and simple and low-cost practices and techniques 
- Acton overall "life cycle" of the innovative technologies 


In the light of all this, it is unavoidable it and almost mandatory to direct efforts, at the same time, 
towards two areas of action: 


i) By working on the frameworks, for to implement "sustainable" practices and simplify the 
controls for implementation to technologies, people and business processes (and/or obtain 
them, effective and cheap, like managed services from specialized third parties, where 
advantageous and practicable) in less staffed companies. 
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ii) By introducing a regulatory schema, imposing a minimum common framework for all market 
players, to incorporate the baseline security controls from the beginning of the life of 
technological components and devices (security is to be evaluated as an integral component of 
any network-connected device). An active role is here hoped and expected from the national 
Authorities, for to create a contribution with security certification, at least for to be considered 
trustworthy those standalone certified devices (before integration in a real computing 
environment). 


Hardwar./ 
manufacturer /integrator, By imposing 
a «sustainable» 
Solution practices 


developer 


/ By regulating ` 
the security 
. «embedded» / 
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SUMMARY 


In the last years, in the manufacturing sector, there has been an increasing diffusion of 
digital technologies, so that the term Digital Manufacturing emerged. These technologies 
have entailed an impact at the corporate level, through the necessary reengineering process, 
and socially, making the customer the protagonist in the production of their purchases and 
enhancing their creativity. This change occurred in response to the reduction of the life cycle 
of products and to the need to meet the requirements of its customers in the best way. 


# 
3D printing is an innovation which, together with digital technologies, has enabled the 


diffusion of digital manufacturing, facilitating the transition from the paradigm of mass 
standardization to that of mass customization. The expiration of some patents related to print 
has encouraged the diffusion of additive manufacturing; now it is increasingly used by 
consumers and enterprises. Since its birth, it has been applied for prototyping, while in 
recent years also for the production of products and spare parts. The sectors where it is 
most widespread are the rail and automative transportation, the aerospace, the medical and 


the artistic ones, with relevant feedback in the Cultural Heritage field. 


3D printing is considered a disruptive innovation because, encouraging the sale of files 
instead of a sale of items, it can optimally solve economic and social problems that have 
been emerging in recent years. #t is considered a technology with strong growth, but it may 
encounter obstacles in its expansion due to the problems concerning the protection of 
intellectual property: file sharing can encourage the spread of non checked 3D files and then 
it leads to the free duplication of digital objects and the consequent violation of intellectual 
property. A natural consequence may be the failure to control the authenticity of the files, 


and then the risk is reflected in the spread of counterfeit and uncontrolled goods. The lack of 
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specific legislation to protect file properties further aggravates the threat of the illegal use of 
the same.#Currently the use of 3D watermarking and encryption of data is an used solution to 
this problem, although an action at national and European level which protects the 
dissemination of unauthorized files is still desirable and new more robust and safe protection 


tools are required. 


Let us consider in more details the 3D printing with protection of 3D data (3D Digital Models) 
activating them at local or remote sites. 

Indeed when objects are copied without permission, there is a distinct possibility of infringing 
third party rights. 

Unauthorised commercial production of products by 3D printing may constitute an act of 


legality infringement by the user of the printer. 


Whilst it seems clear that the manufacture of a whole patented product, for instance by a 
plastic laboratory equipment, will constitute patent infringement, the position regarding the 
manufacture spare parts and their incorporation into patented products, regardless of 
whether they were produced by 3D printing or by traditional manufacturing methods, is not 


as clear-cut. 


Copyright gives a bundle of rights to the rights owner (usually the creator or the creator’s 
employer) to prevent other people from copying (anti counterfeiting), using or exploiting their 
works (works which involved intellectual creation). 

In response to the growing emergence of 3D file sharing and the intellectual property risks 
therein, the research community are developing methods to protect digital data from being 
mistreated. 

These include, for example, mechanisms such as Technological Protection Measures and 
encryption software. There are a range of access control technologies used by those 
(manufacturers, publishers, rights holders) who wish to protect an asset through limiting the 
use of the information or digital device. This protects rights holders from having their 
intellectual property or digital assets copied or converted without permission and is typically 
applied to music and films. These technologies have to be developed however in conjunction 
with all printer manufacturers under a mandate that ensures printers to have the correct 
decryption protocols and software to stream legitimate 3D files. It also allows for enforceable 
liability warranties to be created, giving consumers the confidence that content of high quality 


or providence can be 3D printed. 
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On this line applied research activities are developed by a new START-UP INN-3D, created 
in the framework of the large Firm SESA GROUP in Empoli (Florence), to define and 
implement new Software Tools enabling the Protection of 3D Products and in particular of 
3D Digital Models. A prototype has been developed and tested, using digital marking and 
special security tools to ensure the safe transfer of the 3D Digital Model from the Creator 


(Owner) to the final User (at local and remote sites). A new Patent has been proposed. 


These new developments of 3D Data Protection will have a big increase in the near future, in 
connection also with a reduction of prize of 3D printers (which is expected as a result of 


patent expiries and technological progress). 
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MUSEO NAZIONALE DEL BARGELLO 
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The Museo Nazionale del Bargello is the first National Museum of Italy situated since 1865 in 
the Palazzo del Podesta, one of the oldest buildings in Florence, begun in the mid-thirteenth 
century to host the Podesta and the Capitano del Popolo. In 1574 the palace was transformed 
into a jail, becoming the gloomy fortress where the “Bargello”, the “chief” of police imprisoned 
criminals and traitors. During 3 centuries the Palace was a prison, but in 1858 restorations 
begun, thanks to the discovery - on July 1840 - of the portrait of Dante Alighieri, the greatest 
italian poet, painted by Giotto in the Podestá's chapel. He is depicted in profile wearing his 
typical hat and long red robe, in a crowd of spectators looking at “Paradise”. In 1865 when the 
palace was ready to host the first Italian National Museum, with masterworks from the 
medicean collections, later increased with bequest of famous private collections by Louis 
Carrand, Costantino Ressman and Giulio Franchetti. 

The Bargello is today one of the most important museum of sculpture and decorative arts in 
the world, with Renaissance masterpieces by Brunelleschi, Donatello, Verrocchio, Desiderio 
da Settignano, Michelangelo, Della Robbia, Cellini, Giambologna; it also hosts wonderful 
collections of weapons, ivories, textiles, medals, seals, majolica and paintings. Since many 
years the Bargello stages international exhibitions, often in cooperation with other European 


and American museums. 
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Abstract - While Media Art has evolved into a critical field at the intersection of art, science and 
technology, a significant loss threatens this art form due to the rapid technological obsolescence and static 
documentation strategies. Addressing these challenges, the Interactive Archive and Meta-Thesaurus for 
Media Art Research is developed to advance the Archive of Digital Art. www.digitalartarchive.at Through 
an innovative strategy of ‘collaborative archiving,’ social Web 2.0, 3.0 features foster the engagement of 
the international Media Art community, and a ‘bridging thesaurus’ linking the extended documentation 
of the Archive with other databases of ‘traditional’ art history facilitates interdisciplinary and 
transhistorical comparative analyses. 


As a valuable solution to challenges in the documentation, indexing and research of Media 
Art, the Interactive Archive and Meta-Thesaurus for Media Art Research (AT.MAR) has been 
developed as an innovative strategy for ‘collaborative archiving’ [1]. Supported by the 
Austrian Science Fund (FWF), and conducted at the Department of Image Science, AT.MAR 
is an advanced conception of the Archive of Digital Art (ADA) [2]. Formerly called the 
Database of Virtual Art, this pioneering archive for works at the intersection of art, science 
and technology celebrated its fifteenth anniversary in 2015 [3]. ADA was established as a 
collective project in cooperation with renowned international Media Artists, researchers and 
institutions for the integration of a sustainable exchange between artists, experts and users. 
Comprehensive and open access, ADA is a costfree database. 


HARD HUMANITIES: MEDIA ART HISTORIES & IMAGE SCIENCE 
Over the last five decades, Media Art has evolved into a significant contemporary field. It 
encompasses art forms produced, modified or transmitted by means of the very digital 
technologies that are fundamentally revolutionizing our world—as well as how we perceive 
and interact with images—through globalization, the Internet, social networks, Web 2.0 and 
3.0, and on. Unlike with painting or sculpture, graphic printing or even photography, Media 
Artists make use of emerging technologies that originate from a scientific, military or 
industrial context not only as their media, or image carrier, but have this technology as their 
explicit image-subject as well [4]. Thus, Media Art can take highly disparate forms, and 
includes such genres as bio/genetic, database, digital animation, game, glitch, installations, 
nanotechnology, net art, telepresence, and virtual reality. 

Image Science (Bildwissenshaft in the German tradition), and its sister discipline, Visual 
Studies, encourages a ,reading’ of artistic images that is interdisciplinary, as is essential with 
Media Art. This approach presupposes that scientific work with images must include their 
definition, archiving and a familiarity with a large quantity of images. Though there have 
been a number of historic forerunners to the image science method, most frequently cited as 
the discipline's ,father' is Aby Warburg. Famously intending to develop art history into a 
“Laboratory of the cultural studies of image history” that would widen the field to ,,images 
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[...] in the broadest sense”, by including many forms of images in his iconic Mnemosyne 
image atlas of 1929, Warburg redefined art history as medial bridge building [5] .Yet, 
definitions of the image such as those by Gottfried Bóhm, James Elkins and W.J.T Mitchell 
[6] have become problematic in the context of the interactive, immersive, telematics and 
generative digital image. These challenges have fueled interidisplinary debate as to the status 
of the image with protagonists such as Andreas Broeckmann, Oliver Grau, Erkki Huhtamo, 
Martin Kemp and Barbara Stafford [7]. 


Through the study of MediaArtHistories in the discourses of Media Art the most immediate 
socio-cultural questions of our time are investigated: from body futures and media 
(revolution, to environmental interference, finance virtualization, and surveillance culture. 
While the critical lexicons of classical art history are relatively fixed, the classifying language 
of Media Art is defined with dynamic terminologies that are continually in flux, or so-called 
‘floating signifiers.’ Thus, the forums and catalysts for Media Art rhetoracy take place in a 
vibrant knowledge ecosystem reported in: collaborative projects for database documentation 
supported by institutional and social agencies; international festivals with peer reviewed 
awards and globally publicized interviews; and new literatures published by leading scientific 
and university presses [8]. Yet, despite such worldwide recognition, programmes for 
documenting the ‘art of our times’ continue to be met with serious challenges within the 
memory institutions of our societies. As Media Artworks frequently have functionalities 
across variable media substrates, and these constituted by the latest technologies as well as 
characterised by a rapid obsolescence, the work of Media Artists complicate both object- 
oriented preservation methods as well as static indexing strategies. Consequently, artworks 
originating even just ten years ago can often no longer be exhibited. As debated since the 
1990s, museums rarely include Media Art in their collections, and those that do struggle to 
sustain finance, expertise, and technology for the preservation of artworks through strategies 
such as migration, emulation, and reinterpretation [9]. Further, that Media Artists engage the 
most contemporary digital technologies leads to the production of artworks that are 
necessarily “processual,” ephemeral, interactive, multimedia-based, and fundamentally 
context-dependent [10]. 


Since the turn of the Third Millennium, there has certainly been evident promotion of 
conferences, lexicons, and platforms in the endeavour to document Media Art. It is 
specifically the subject of the MediaArtHistories conference series, which with its premier in 
2005 represented and addressed the many disciplines involved in the then emerging field [11]. 
A number of preservation projects have also been established. While many continue to exist 
online, each either lost key researchers, had funding expired, or was eventually terminated 
[12]. And as recently expressed in an international declaration [13], signed as of 2016 by 
more than 450 scholars and leading artists from 40 countries, there is an urgent need to create 
a stable international platform of interoperable archives. Yet, even with such progress in the 
study of Media Art, programmes for documenting this ‘art of our times’ continue to be met 
with serious challenges within the memory institutions of our societies. Indeed, it is no 
exaggeration to state that we continue to be threatened with a significant loss of this critical 
art form, both in the archives of art history and for future scholarship. 
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MEDIA ART (R)EVOLUTION AND THE ARCHIVE OF DIGITAL ART 

Since the year 2000, ADA is one of the most complex research-oriented resources available 
online as a platform for both scientific information and social communication. Hundreds of 
leading Media Artists are represented by several thousand documents, with more than 3,500 
articles and a survey of 750 institutions of media art also listed. Besides the artists, there are 
also more than 250 theorists and media art historians involved in making ADA a collective 
archiving project (Fig. 1). 


Fig. 1. Archive of Digital Art, screenshot (detail, Community Light-Box) 
<https://www.digitalartarchive.at/nc/home.html>, accessed 4 March 2018. 


Because of the singular structure of the art form, a defining strategy for the Archive of Digital 
Art is that of an “expanded concept of documentation” [14]. The documents on ADA that 
represent the artists there archived include: biographical and bibliographic information about 
the artist, their inventions, awards, and statements; exhibitions, and publications; graphic 
images of the installation of the artwork; digital images of individual artworks (exhibited, in 
process, and in all its varying iterations); information on the software and hardware 
configuration; technical instructions; type of interface and display; video documents 
(interviews, presentations, symposia); references and literature about the artists; information 
about the technical staff; institutions; and copyright. 


A system of online community membership for ADA allows artists and scholars to upload 
their own information, with a gate-keeping policy that the ADA advisory board reviews 
applicant qualifications and makes member selections. The system offers a tool for artists and 
specialists to individually upload information about works, people, literature, exhibits, 
technologies, and inventions [15]. Over the last fifteen years some 5,000 artists were 
evaluated, of which 500 fulfilled the criteria to become a member of the ADA. From the 
beginning, the long-term goal of the project was not simply the documentation of festivals, 
awards or similar events, but a scientific overview with the respective standards of quality. 
Members have to qualify with at least five exhibitions or articles about their work, or, 
alternatively, can be suggested by the board. 
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DOCUMENTING MEDIA ART: IMPLEMENTING 2.0, 3.0 FEATURES 
For the Archive of Digital Art (ADA), the first online collective archive that is both scholarly 
and social in either art history or media studies, documentation and access are not understood 
as static concepts, but as a process that integrates a continuous exchange between users, 
scholars and artists. With an open access policy that provides users with an active role and 
that supports accessibility, ADA is more likely continue to be an up-to-date as well as a 
lasting resource. An essential aspect of its Interactive Archive and Meta-Thesaurus for Media 
Art Research (AT.MAR) was thus to transfer ADA into a Web 2.0 environment and open it 
up on the ‘retrieval-side’ by making the data available and easier to share for users, and on the 
‘archivist-side’ by allowing contributions of diverse individuals in order to facilitate a 
collaborative and more balanced preservation practice. Newly innovated ADA features 
support the group engagement and foster motivation. A messaging system and “News” 
section allow archive community members to interact with peers and announce upcoming 
events. Contribution monitoring and a function for colleague “following” provide updates on 
the research and activities of other Archive members. And collaborative processes of peer- 
reviewing and content curation, integrate the member community’s decision-making and 
agenda setting into ADA itself. Contributions can be seen in the “Works” section of every 
scholar and artist on ADA, where the Archive features enable members to collect 
“Descriptions and Essays” about their artworks, as well as information on “Technology,” 
“Literature,” and “Exhibition and Events.” A process of peer-review performed by the ADA 
member community guarantees the quality of these contributions, with all the “Latest ADA 
Updates” visible to members on the homepage after login. Individual contributions, once 
peer-reviewed, are automatically referenced and made accessible to all users, whether ADA 
community member or online visitor to the Archive. Contribution visibility is measured not 
only in web links, page hits, and citation statistics, but also exemplified by the above 
described peer assessments internal to the ADA, in a disciplinary as well as interdisciplinary 
networking that builds the standing of Archive members within their international 
professional community. 

Members also engage in selecting a monthly-featured artist or scholar, a profile about who is 
published on the ADA homepage, social media, and through web newsletters. This “Featured 
Artist/Scholar” introduces ADA visitors to artists and scholars distinguished by their peers; 
allows Archive members to commemorate achievement within the discipline or recognition 
within the community; and supports active participation in content direction. Additionally, 
ADA’s “Light Box” (Fig. 1) feature is both scholarly and social. Promoting the comparative 
analysis of Media Artworks on the Archive, this tool permits community members to 
assemble individual arrangements from the extended documentation of images, texts, and 
videos on ADA. These “Selected Items” can then ‘enlarge’ and ‘overlap’ so that relevant 
image details can be compared and analyzed. Textual notes can be added and “Exhibitions” 
saved on a visual pin board of “My Screens” for further research. These “Light Box”-based 
exhibitions of ADA content by community members are then publishable as an “Online 
Exhibition,” visible to all users, and accessible for a wide variety of applications from 
scientific or art-based research, to science, education and public outreach. ADA promises 
many potential affordances as an online collaborative archive, including expanding data 
beyond that which any single institution or even cross-institutional research team could 


122 


compile; increasing the high quality of data that originates directly with artists and scholars in 
the field of media art; cultivating the various viewpoints of the global community that 
contributes to the archive; and developing this scholarship through a system of checks and 
balances by Archive community members. Features such as “Works” contribution and peer 
review, “Featured Artist/Scholar of the month, and “Light Box” peer review all enhance the 
interpersonal relationships of ADA community members and foster exchange. 


INDEXING MEDIA ART: THE BRIDGING THESAURUS 

Keywording is bridge building! And for the ‘bridging thesaurus’ of the AT.MAR project, the 
intent to establish a linguistic framework that allows for the classification of the aesthetics, 
subjects and technologies of artworks, directs the process of individual concept and term 
selection. To achieve a comprehensive overview of the knowledge domain of Media Art, but 
also a manageable one, this vocabulary is kept limited to around 400 terms. This constraint 
increases the usability of vocabulary terms and insures an accuracy for indexing practice, 
which is particularly crucial with ADA as it the community members themselves who carry 
out a significant part of the indexing. Central to the construction of the ADA controlled 
vocabulary is the logical concept of terminology structure based on a classification strategy 
that will allow users to index various levels of meaning relevant to the Media Art knowledge 
domain. In relation to other vocabularies, ADA “Keywords” have a unique hierarchical 
schema based on a categorical triad of “aesthetics”, “subject”, and “technology”. This top-down 
distinction of categories allows for the contextual specification of vocabulary as well as for 
the conceptual analysis of these levels by users: 4esthetics: In accordance with the dominant 
understanding of Media Art in the scholarly literature, and ‘relatives’ of this field such as 
digital or electronic art, the “aesthetics” category encompasses a broad scope of terms ranging 
from phenomenological observations such as ‘immaterial’ to ontological qualities such as 
‘site-specific’ and ‘object-oriented’. Subject: The “subject” category encompasses 
iconographic terms established in art history and Media Art Histories, as well as concepts that 
enable both descriptive and interpretative approaches to the subject of works. In regards to 
term quantity, this category is the most comprehensive [16]. The ‘subject’ category includes 
13 subcategories like: ‘Body and Human’, ‘Entertainment and Popular Culture’, ‘Magic and 
Phantastic’, ‘Media and Communication’, ‘Nature and Environment’, ‘Technology and 
Innovation’, ‘Power and Politics’, ‘Psychology and Emotion’, ‘Religion and Mythology’, 
‘Science and Knowledge’. Technology: The ‘Technology’ category was adopted from that 
originally developed for the Database of Virtual Art (DVA), which later became the Archive 
of Digital Art, and enhanced by subcategorization ‘interface’ and ‘display,’ as well as terms 
encompassing ‘traditional’ image-carriers such as ‘painting’, ‘print’ or ‘book’. The Resources 
of terms and concepts used in the development of AT.MAR and which define the very 
foundation of this controlled vocabulary, include (1) ‘traditional’ art history vocabularies as 
well as (2) Media Art databases, (3) festivals, and (4) literatures: 

(1) The ‘traditional’ art historical vocabularies cited were those most widely accepted 
scientific tools used for the description, linkage, and retrieval of images in art history. These 
included Iconclass, an alphanumeric classification scheme designed for the iconography of 
art; the Art and Architecture Thesaurus (AAT) and the Warburg-Index, an index of 
iconographical terms. 
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(2) Databases were selected for AT.MAR, includine The Dictionnaire des Arts Médiatiques, 
GAMA keywords, the vocabulary of the Daniel Langlois Foundation, and Netzspannung. 
Each of these vocabularies reflects the explicit practical affordances and implicit ideological 
assumptions of the institution that advanced it. 

(3) Further, as festivals are central to the media art scene as forums and catalysts for the 
contemporary discourses and innovative technologies of media art, the project team took 
account of an international range of festival materials such as official publications and 
professional interviews. Festivals reviewed included, among others, Ars Electronica; Dutch 
Electronic Art Festival; European Media Art Festival; Festival Internacional de Linguagem 
Electrónica; Inter-Society for the Electronic Arts; Microwave Festival, Transmediale.. 

(4) Research literature was evaluated on the basis of its indexes, that “map” the most valuated 
topics in the field. Important innovations such as, “interface” or, “genetic art’ were considered 
along with keywords that play a role in traditional arts— such as ‘body’ or ‘landscape’— with 
a bridge-building function. 


FUTURE MEDIA ART RESEARCH: THE GOTTWEIG COLLECTION 
To support the cross-cultural, inter-disciplinary, and trans-historical comparative analyses of 
the Media Artworks on ADA, the keywords of its ‘bridging thesaurus’ are further applied to 
artworks from other social contexts and historical periods. Through AT.MAR, ADA is now 
linked with the Góttweig Abbey Graphic Collection (Fig. 2). Góttweig Abbey, founded in 
1083, holds 30,000 prints as well as a library of 150,000 volumes in one of the most 
comprehensive private collections of mostly Renaissance and Baroque engravings. With 
acquisitions first recorded in 1621, the collection was systematically expanded during the 
Abbotship (1714-1719) of Gottfried Bessel. In cooperation with Góttweig, the Department of 
Image Science conducted the digitization of the collection [17]. 


Fig. 2. Góttweig Monastery Graphic Art Collection Online, screenshot, 
<http://www.gssg.at/>, accessed 4 March 2018, O Department for Image Science. 


The graphic and textual works of the online Góttweig collection, document subjects from the 
‘representation of knowledge’ and “history of science,” to “architecture” and ‘fashion,’ ‘optics’ 
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and “panorama.” Thesaurus keywords navigable as “Hierarchical,” “Alphabetical,” and “As 
Cloud” , support and stimulate users to bridge the “traditional” artworks and the Media Art of 
ADA, providing complex image resources for a richer analysis of Media Art (Fig. 3). 
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Fig. 3. Media Art Research Thesaurus, screenshot 
(detail, comparative analysis), accessed 4 March 2018. 

CONCLUSION 

The innovative methodology developed through the AT.MAR project will foster the 
documentation, indexing and research of Media Art on the Archive of Digital Art in a context 
of multiple histories of art, science and technology. Thesaurus categories in aesthetics, subject 
and technology bridging ‘traditional’ art forms with Media Art support the tracing of hybrid 
qualities in these artworks, as well as historical correspondences and conflictions. Through 
collaborative visual tools that include a Light Box and semantic links, a global community of 
artists and scholars may conduct research and perform clustering analysis or comparative 
study. That the thesaurus connects Media Art with art history, and neither isolates these fields 
from one another nor includes only contemporary terminology, increases the usability of the 
Thesaurus for the humanities. For the future of ADA, further goals are to document works 
within a context of complex information and, at the same time, allow users to find specific 
information readily. Beyond using analysis using the Briding Thesaurus, which shows, for 
example, virtual and immersive art’s reminiscences to its predecessors in the panorama and 
laterna magica, Media Art documentation should also include questions of gender, track the 
movement of technical staff from lab to lab, technical inventions pertaining to art, as well as 
public and private funds allocated to research. By advancing from a one-way deposit of key 
data to a proactive process of knowledge transfer, the archiving of Media Art becomes a 
resource that facilitates research on Media Art for academics, experts, and students. 
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Introduction 
In recent decades, modern technologies have pervaded every activity of Cultural Heritage with an 
incredibly growing pace. All the areas have benefited from that, starting with knowledge (with the 
availability of increasingly sophisticated archaeometric analyses), conservation (within the activities of 
diagnostics, restoration, preventive conservation, monitoring), fruition and valorization (through the 
multimedia-ICT and the lighting techniques), security and safeguard (thanks to a variety of electronic 
sensors implemented against theft, vandalism, environmental degradation, catastrophic events, transport 
damage), to documentation (with the precious digital imaging systems) and archiving (with the databases 
and the software to access the archives of documents and images). 
A new generation of specialists have emerged that have adapted their specific skills to the needs of CH, 
professional figures who find employment in museums, in art and restoration exhibitions, in public and 
private scientific laboratories, in the archives of CH. The Cultural Heritage, traditionally considered a land of 
humanistic knowledge and competence, has become a context of interaction, integration, meetings, 
comparisons between art historians, architects, archaeologists, restorers, together with chemists, 
physicists, biologists, geologists, engineers, technologists, to the full advantage, in the broadest sense, of 
the CH. 


The Imaging techniques 

Among a so wide variety of new technologies involved in this new mega-system, here we want to focus to 
the imaging techniques. 

The introduction of Film Photography, in the early decades of the 19% century, although initially only in 
black and white, introduced a real revolution. Now works of art could be not only observed, drawn, 
painted, but recorded in an objective document, stable, ready to be easily examined by several specialists, 
in the same time as well as in a successive moment. 

Documentation, diffusion of images, possibilities of comparison have expanded as never before. With the 
development of Color Photography, towards the end of the same century, the photographic document has 
grown furthermore in accuracy of reproduction: color is a fundamental parameter for works of art and 
paintings in particular, although treatises of art history have continued for decades to illustrate figures in 
black and white, probably for cost reasons. 

In the final part of this paper, we will come back to photography with the topic of the digital images, to 
explore the enormous potentiality of this photographic technique developed in later times. 

Among the variants of photography, the Raking Light Technique is worth of a special attention. Micro and 
macro deformations of the surface, induced by degradation processes active along centuries, resulting in 
swelling, flecking, loss of material, formation of lacunas, efflorescence and other similar alterations, 
inevitably have affected any ancient artifact. With a photograph taken in raking light, any alteration of the 
object surface is amplified; its location and the extent of the phenomenon are highlighted. 

Raking Light Photography is equally useful for studying the techniques of execution, in a painting, for 
instance, thickness and orientation of the brushstrokes; in a fresco, the so called ‘day's works’. 
Under the Optical Microscope, thanks to magnification, imaging offers the expert an extremely powerful 
tool to enter within the constitutive material of an artifact, revealing a quantity of information on 
morphology, colors, sequence of layers in the Cross section of a sample. Finally, with the advent of the 
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Scanning Electron Microscopy, SEM, (3rd decade of the 20th century), the study of the surface morphology 
sample has reached the highest levels. 

At the end of the 19% century a new extraordinary imaging technique is invented: Radiography. 
Not only the external appearance of an object can be documented, as it had always been, but its interior 
too, revealing material structures inaccessible to the eye. X-rays, a radiation with infinitesimal wavelengths 
(A = 10 -10?nm), much shorter than the visible light (A= 700 nm/red - 400 nm/ violet), competitive with the 
atoms size (about 0,1nm), allow to cross solid bodies, depending on their thickness and composition. 

In the second decade of the 19" century (1916-18) some more advanced museums begin to radiograph art- 
objects in a systematic way. Discoveries follow each other in an enthusiastic way: unthinkable artist's 
pentimenti are brought to light, multiple paintings overlapped on the same panel or canvas are revealed, 
internal pivots are identified inside wooden statues. 

Radiography is the first major ascending step in the world of diagnostic imaging. Once undertaken, this 
new road will be expanded with the use of even more penetrating radiations, gamma rays with 
wavelengths in the range «10?nm, but above all with greater energy. 

They are capable of crossing much larger thicknesses than those of panel paintings, and materials much 
more compact than wood: i.e. metals, stones. This opens the way to identify and locate elements inside 
bronze, marble, stone sculptures, to measure thicknesses, identify joints, welds, diagnose the presence of 
internal cracks and other discontinuities; practically, the possibility of acquiring a wide range of information 
both on the structure of artefacts and their state of conservation. 

Going even further in the ability to penetrate very thick structures, in very recent years a decidedly 
futuristic application has appeared: the use of ultra-high energy radiation, made of Muons, subatomic 
particles with a charge similar to that of electron and mass about 200 times higher. Muons are highly 
unstable particles with a very short life time, that, however, can reach the earth's surface coming from the 
atmosphere boundaries, originated from cosmic rays. Recently they have been successfully employed 
looking for new internal chambers in the Pyramid of Cheops. They are going to be applied to the Dome of 
the Florence Cathedral in search of possible internal metal chains. 

Going back to more accessible technologies, other radiations close to the visible range, overlook the art 
world: the Ultraviolet (A~ 400nm - up to the XR), in use since the 2"° decade of the 20^ century, and the 
Infrared (A= 0.7um - 0.4mm), from the 3" decade of the same century. 

Also these radiations are recorded on photographic film. They subdivide in a number of specific techniques 
according to the taking conditions (source of the radiation, filters to select the source, types of film to 
record the images, etc.). 

The Ultraviolet is distinguished in Reflected UV and UV Fluorescence. A reflected UV image is directly 
recorded on the film while in the second case the fluorescent visible image excited by UV is recorded. 
Reflected IR is quite similar to Reflected UV: the reflected IR image is recorded on a special film. Initially, 
various types of black and white films were used, the only available at the beginning of these experiences. 
For decades the availability of only black and white films has been a limiting factor that penalized the above 
techniques, especially UV Fluorescence which manifests as a color image. As soon as Color Photography 
became available it was possible to record in their fullness the specific fluorescence-colors depending on 
the nature and the age of the materials of the object. 

Later on (over the 70s of the 20th century) false colors infrared films were introduced, capable of mutating 
an Infrared-Red-Green image (Blue is excluded by a filter) in a 'false' Red-Green-Blue image. Thus, a new 
technique was born, the /nfrared False Colours. 

The use of these 4 techniques is based on the specific way UV and IR radiation interact with the constituent 
materials of the objects, which is different from that of light. By comparing the UV or IR images with those 
obtained with common light, very important information, although not foreseeable a priori, can be 
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acquired on the nature of materials and figures not visible or not differentiable by eye. 

But there is a further possibility. Depending on wavelength, the infrared radiation can penetrate more or 
less into matter. This makes it possible to differentiate in a painting, also materials (or figures) located 
below the paint layer, such as preparatory drawings, pentimenti, etc.. 

Starting from the '80s, with the development of opto-electronic technologies, photographic films were 
replaced by cameras, powerful instruments, able to go beyond the visible, in the infrared region. In this 
range some of the surface paint layers become more transparent. This is how Infrared Reflectography was 
born, initially realized with cameras and later, with more sophisticated Laser-Scanners in the Visible- 
Infrared region. IR Reflectography and IR-VIS Laser Scanners are today the techniques of excellence to 
explore the hidden layers of a painting, especially the preparatory drawings, with striking results that have 
unveiled the evolution of an artist in his creative activity. 

With the VIS/IR Laser Scanner it was possible to reproduce the /R-False Color-Photography with a precision 
that amazes. But today, IR-False-Color can also be realized with some special digital photo-cameras. In an 
analogue way also a Digital False-color-Ultraviolet technique has been implemented. 

We started this extraordinary path of technological innovations applied to Cultural Heritage with Film 
Photography. This technique, this period, are now almost extinct. Digital Photography has penetrated in 
any context. Phones, tablets, digital cameras of recent generation are available to all, at affordable prices. 
Taking digital photographs is nowadays a diffuse practice. The advantages are too many and too 
suggestive. You can take photos at will, evaluate, discard the badly-done, save the good, perfect your takes 
with software that allows you any processing, finally archive on digital supports without space problem; if 
you want, you can print your photos with excellent colour renditions. 

To complete this overview we want now to come back to the normal digital photografy but that at an 
excellence level, the so called High Definition Digital Photography (HDDPh), which can only be realized with 
professional equipment by specialized operators. The role of this special digital Photography has been 
remarkable, not only for storing the reproductions of the art-objects in Museums archives (which other 
document, if not a HDDPh, could be more accurate and reliable?) but also as a powerful tool for studying 
artworks at disposition of art-historians, archaeologists, architects. Sitting comfortably in their office, they 
can examine their images on a monitor, enlarge them at will, highlight any figurative or material detail 
undetectable by other means: in practice, a boundless possibility of study. 

In the field of conservation, not to mention the ease of documenting a restoration in progress in all of its 
steps, or monitoring the evolution of the state of an artefact post restoration, we want to focus here on a 
powerful application, rich in promising developments: the Virtual Digital Restoration. | am well aware, from 
personal experience as well, how this approach is still considered with prudence by experts (art-historians, 
etc.). The hesitation, which | consider unfounded, comes from idea that it could become competitive with 
the ‘real’ restoration. On the contrary, let's think about the advantages that this availability could offer a 
public or private restoration organism. For instance, once the canonical operations (cleaning, consolidation, 
filling) were finished, it would be possible to simulate in advance a series of possible solutions for the final 
step of restoration, the so-called aesthetic restitution, without intervening on the object with choices that, 
in retrospect, might be not the most appropriate. This not only valid in the restoration area but for example 
for deciding the way to expose an artwork in a Museum, or for setting an exhibition, and so on. A variegate 
opportunity, about which it is convenient that the stakeholders reflect seriously, to evaluate, without 
prejudices, the undoubted advantages. 

To illustrate adequately each of the numerous imaging technologies for C.H. developed in the last few 
decades, a treatise would be needed. With the present contribution we tried to give, at least, an overview 
about the crucial role they are playing today in favour of conservation, knowledge and valorization of 
Cultural Heritage 
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ABSTRACT 

Light is the connection between objects and human being and it assures visual perception. 
Changing light the colours and the feeling of painting change. 

The new lighting source (Led) offers many advantages and also a wide variety of different 
lighting spectrum not expected in the past years. 

But, very import to know the "language" of visual art. 

Wrong light can damage the quality and the aspects of the artwork. 

A Important recent research ( 2015) by National Research Council inside the laboratory of 
Opificio delle Pietre Dure in Florence have investigate the changing of perception with the 
different light [1] 


fari A uum aci tatas 


“Sant’ Agostino nello studio” 
Sandro Botticelli 


This painting ware lighted with different 
type of led source: Ra 84 3000 K and 
4000 K, Ra 98 3000 K and 4000 K, Ra 
92 3500 K and 4200 K. 

For the test 25 people are invited : 
experts in art and other professional 
skills. 
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The preference were for the lighting sources with Ra 84 and not Ra 98 as expected. 

In the Ra 84 the presence of red is more inside the spectrum and the incarnate and other details 
were more impressive. 

What is the Ra? [2] 

The colour rendering index (CRI) defines the ability of a light source to identify colours, and is 
measured on a scale of 1 - 100. 

On this scale a rendering of 1 is monochromatic light, and a rendering of 100 is natural sunlight, 
so you can think of the scale as a measure of the quality of light produced by the source. 

For example, have you ever stood under an old street light at night and wondered why the colour 
of a car or your wheelie bin doesn’t seem the same as it does during the day? Although the lamp 
used in the street light gives off a lot of light, the CRI is very low so the appearance of any 
colour is transformed. 

What does it mean? 

Lighting is not an exact science. 

It is not enough to study the correct quantity of light, its Ra, and its qualities. 

There is a deeper relationship in the act of vision and its reaction to a work. 

Importance of the visual experience. 

Each work has its own specific character. 

The luminous reading of a painting should have the ability to recreate the original atmosphere, 
the luminous climate present at the creation of the work. 

Importance to analyse the psychophysics of vision. 

Obviously there are different schools of thought but also related to different types of works of 
art. 

The light in the museum is like the light for the stage: it creates a experience. 

For example the lighting designer Scott Rosenfeld [3] in the «Colour Field Gallery» of the 
Smithsonian American Art Museum in Washington: «Our objective when lighting our modern 
galleries, especially the one dedicated to Colour Field painting, is to help artworks appear as a 
natural extension of the white wall. The lighting should subtly ,pop‘ works off the wall without 
making them look ,spot lit*. If the relationship between the wall and the work is perfect, 1t will 
appear almost to float on the wall.» 

The paintings of the Renaissance are differents, tells stories, every painting is full of symbolic 
references, in many painters light reveals the truths and transports the spectator into his world. 
Interesting exhibition have analysed these relationships : "In luce, storia arte e simbolismo 
dell'illuminazione " [4] was set up by the conservator of the Ethnographic Museum, Tiziana 
Ribezzi, with the collaboration of Lucia Stefanelli from the State Archive of Udine and Lucio 
Fabi (2015) 

The Earlier Mona Lisa exhibition [5] in Shanghai hit me a few years ago (2015) in this logic of 
research and attention to lighting. 

The whole exhibition is focused on the scientific and historical path of the discovery of the 
painting, a long visual and interactive journey. 

In the last room, we enter, in a dark room containing the painting. 
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The exhibition is transformed: from a scientific event to a theatrical event. 

The visitor is suddenly transported into a theatrical experience. 

The painting slowly comes alive, slowly unveiling (about 2 minutes) all the elements of the 
artistic composition. 

The EMT European Museum Technology with the Senior Engineer Dieter Colman have 
designed a complex lighting system with: 

24 LED’s, dimmer controlled by DMX ( the dmx 512 is a light control protocol) 

and dozens of Precision beam (microspot) from 5 — 60 degrees with the 
possibility of adjusts pan, tilt, spot and flood (CRI 97 3000K) 


All products are LUXMAN. 


With a Netswitch device the lighting and the animation starts 
automatically in the morning and stops automatically at night. This also 
allows local people to use a phone or tablet or laptop to turn on the 
lighting (with the animation) for a private showing outside exhibition 
opening times. 


The images show the lighting sequence. 


I would like to conclude with a quote from Marcel Duchamp: "It is the 
spectators who make the painting"....An interesting reflection. 
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ABSTRACT 


This paper presents “Renaissance Experience: Florence and Uffizi”, the evolution of 
“Uffizi Virtual Experience - da Giotto a Caravaggio”, the immersive and interactive 
digital exhibition created in Italy, which has brought to Milan the masterpieces of the 
Florence Uffizi Gallery with high-resolution digitized images during 2016. 
“Renaissance Experience: Florence and Uffizi” has been produced by Virtultaly, a 
Centrica innovative startup spin off company, which conveys a new approach to 
edutainment, where cultural recreation is achieved through an immersive educational 
and entertaining experience. 
“Renaissance Experience: Florence and Uffizi” has been opened January 20" 2018 in 
Leipzig, at Kunstkraftwerk, a post industrial location that since 2016 is devoted to 
immersive art. It will be possible to experience this exhibition in Leipzig till August 
2018. 

The paper describes this new digital exhibition, technologies involved and dwells 
upon the technological, sociological and cultural aspects of this new way of presenting 


and promoting cultural heritage in our society. 
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Abstract 

The recovery of the history of humanity goes through the comprehension of ancient testimonies 
handed down to us in artistic representations. 

The innovative tool par excellence is the technologies. 

Smarticon is an Italian patent for method in charge of studying historical analysis (originated from 
the morphology of the concept), of examining the phenomenology analysis (which results in the typo- 
logy and exceeds it) and to the comparative research (which allows to arrive at the hermeneutics ). 

The comprehension of the transitory meaning of each event is entrusted to the historian who, by re- 
composing the phenomena, is also able to codify the comparative meanings. 


INTRODUCTION 


Globalization in the world of art: needs and solutions 

With the rooting of a mentality increasingly oriented towards globalization, the need to adapt to mod- 
ern times also consolidates in the world of culture. 

The communication "towards an integrated approach to Cultural Heritage for Europe" (European 
Commission Brussels, 22.7.2014 COM (2014) 477 final: ec.europa.eu/culture/policy/culture-policies/ 
cultural-heritage_en.htm), has underlined how the Cultural Heritage sector assumes strategic relevance 
also from an intersectoral point of view and how such peculiarity is able to nourish new areas and busi- 
ness models that harbinger employment and professional growth. 

These new models also represent the instrument of excellence to promote sustainable development, 
the strengthening of scientific and technological bases and social cohesion. 

To achieve the aims, the fundamental need is to create an eco-system that is able to create a pole of 
convergence between synergies of different nature, which also interact with those belonging to different 
cultures. 

In fact, innovation and development are stimulated by comparison, sharing, integration, implementa- 
tion and modulation between different competences and systems. 

The eco-system represents the essential prerequisite in order to create the conditions for the competit- 
iveness of the Cultural Heritage industry and for the pursuit of strategic objectives. 


Smarticon: method for the enhancement of Cultural Heritage both from a historical 
and economic point of view. 

To create an eco-system dedicated to the Cultural Heritage sector means to enable a platform for the 
organization and fruition of the assets and resources from which knowledge and enhancement arise. 

It is a fundamental condition that the conservation of Cultural Heritage derives both from the point of 
view of preservation of the material that constitutes the good, and from the point of view of the recovery 
of information that derive from the cognitive heritage recovered by experts and attributable to the arti- 
fact in question. 
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Through the harmonization of the cognitive heritage it will be possible to access to the comprehen- 
sion of the complex scenario from which the good originates and aspire to its codification and enhance- 
ment under the historical, iconic and economic aspect. 


The Cultural Heritage projected towards a planetary humanism 

“Apparently, the criticism (intended as interpretation and evaluation, the round is an explanatory in- 
sert of the writer) opposes to the comment (intended as exegesis, erudition: a set of principles and criter- 
ia aiming to the coding of the original), as the analysis of a visible form to discover a hidden content. 
But being such a form the one of a representation, the critics can not analyze the language if not in 
terms of truth, exactness, property or expressive content”. (M. Foucault, Le mots et les choses, Paris 
1966; it: Le parole e le cose, Milan 1967, p.95). 

The coding of the message expressed in the art object will allow us to examine the multiplicity of 
“languages” that determine its individual forms of expression. 

Apparently without reciprocal connections, they will reveal the rhetoric of a plurality of interconnec- 
tions that testify how man's nature places the human being at the centre of a planetary humanism, where 
the space between the ethnic and social differences assumes an almost trifling value. 

The non-obvious relationships will be revealed through the analysis of the tropes, that is, of the dif- 
ferent relationships that the "words" of a "discourse" may assume within the same representative con- 
tent. 

The eco-system thus conceived will re-evoke the truth and the balance of things, until it re-establish- 
es the virtues of the discourse and highlights the concomitances between apparently different cultures. 


Religions in the world 

The great historian Arnold Toynbee highlights that there is no culture or civilization that has ab- 
stained from any form of religious belief. 

Therefore the nature of the human being is that of the homo religiosus which resulted in traditions 
that have demonstrated not only the need for “truth”, but also the ambition to pursue that “germ of 
eternity ” that dwells in each of us. 

Victor Hugo wrote: “We must say it and say it again, it is not a need for novelty that torments the 
spirits, it is a need for truth; and it is immense". 

In fact, the numerous forms of religious expression testify the common prospect of generating hope 
in man and of projecting his soul towards the research of absolute Truth, the harbinger of eternal salva- 
tion. 

The religions of the world, therefore, can be compared to the trunk of a millennial tree from which 
many ramifications and new sprouts arise, which however are nourished by the same lifeblood. 

Nathan Sóderblom (1866-1931) Lutheran archbishop, theologian and historian of religions, focused 
on the experience of Man who interprets the divine through the multiplicity of religious experiences 
that, on the basis of our concept of "religious globalization", can be interpreted like the ramifications of 
our tree. 

The comprehension of this complex scenario therefore lies in the definition, interpretation and related 
application which derives from the analysis of the tropes. 

One of the aspects that makes essential the need to valorise Cultural Heritage consists in the compre- 
hension and, therefore, the codification of how the homo religiosus has interpreted, for example, mana, 
that spiritual power, that "... vital force intrinsic in things for which they flourish, grow and bear good 
fruits. "(V. Grottanelli, Ethonologica, L'uomo e la civiltà, III, Milan 1965, p.404). This force represents 
the perception of a reality that goes beyond the one of this world, from which the divine sentiment ori- 
ginates. It is worth remembering, then, with Mircea Eliade, one of the greatest historians of the religions 
of all time, that “through the experience of the sacred, the human spirit has grasped the difference bet- 
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ween what is revealed to be real, powerful, rich and meaningful, and what is devoid of these qualities: 
the chaotic and dangerous flow of things, their apparitions and their fortuitous disappearances empty of 
meaning ... In short, the “sacred” is an element in the structure of consciousness, and not a stage in the 
history of consciousness itself. At the most archaic levels of culture, living as a human being is in itself 
a religious act, because feeding, sexual life and work have sacred value”. (Histoire des croyances et des 
idees religieuses, Paris 1976; tr.it. Storia delle credenze religiose, Florence 1979, page 7). 

In the context, the artistic expression plays a fundamental role, since it represents the "language" th- 
rough which these ideals were expressed and thanks to which today we get reminiscence. 

Nevertheless it is reasonable to affirm that iconography in artistic representations interprets the critic- 
al aspect of the concept to which it refers; but also the comment related to the encoding of the content 
hidden in the artifact, to which is entrusted the task of passing on (through the representation), the lan- 
guage of truth, of accuracy and the expressive content of a past era. 

In this complex scenario Smarticon arises as a scientific method in charge of the “hierarchical” order- 
ing of the information and its relative coding. 

It is from this synergy that the ecosystem of cultural heritage comes to life and nourishes itself, pro- 
jecting itself towards a planetary humanism, aspiring to that innovative reorganization of knowledge 
which represents the fundamental passage to trace the iconographic parallelisms and socio-cultural 
meeting points between different civilizations: the comprehension of the transitory propaedeutic mean- 
ing to the encoding of comparative meanings. 


Paradoxically, it is almost “accessory” how a god is idealized; because the essential point of the 
question of the religions in the world is how mediation leads to eternal salvation and how to convince 
oneself that this condition corresponds to the truth of things. From this point the search for evidence 
that may lead to testify, in tangible form, the belief' in a religious phenomenon, arises. 


The religious message in art: parallelisms and iconographic convergences between the 
sacred and the profane 

A considerable quantity and variety of Cultural Heritage, through the symbolism intrinsic in the arte- 
facts of art, interprets the religious phenomena of every age. 

The research for the divine is the foundation that joins man in the religious experience, which tran- 
scends the earthly life and which nourishes the spirit through the hope of eternal salvation. 

Therefore, in the modalities of interpretation of the religious phenomenon, the patrimony of humanity 
is materialized, which it expresses itself and is handed down through the communication tool par excel- 
lence: the iconographic symbolism in the artefacts of art. 

This awareness imposes a great challenge: to be able to face the need to perceive by intuition and 
grasp the “truth”; as the codifing of “languages” implies the comprehension of the modalities through 
which homo religiosus of every age deals with, interprets and solves the enigmas of life and the disap- 
pointments of the human soul. 

However, the theoretical concepts get exponentially complicated in the moment in which arises the 
need to translate the theoretical passage into the concreteness of the image; since iconography is not al- 
ways explicit and, very often, the allegorical meanings are hidden behind ancient customs that are for- 
gotten today. 

Smarticon allows to arrive to the comment intended as an analysis of the visible form, aimed at dis- 
covering the content hidden in the artefact through the discovery of connections recovered with the ana- 
lysis of the tropes. 

Renaissance art, for example, testifies how homo religiosus has been able to translate into exquisitely 
profane background themes, profound Christian concepts: because it is through the beauty and harmony 
of forms that one reaches the supreme “reasoning” which is proper to the idea of divine perfection. 
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The Christian message contained in them can be codified through the comparison of symbolic cor- 
respondences. 

The hermeneutics of language in art with a profane background can also be interpreted through the 
comparison of the hierophany (presence and revelation of a sacred or divine element) which expresses, 
through the artistic representation, the power of the good that overtakes the forces natural. 

For this reason, the hierophany intrinsic in the representation of an object of art becomes a pretext to 
express the supreme theophany, as a sensitive manifestation of divinity. 

Hermeneutics has the task of unveiling the comprehension of the “latent” message, conceiled beyond 
the image and hidden behind the "occulting veils" of time. 

In art, both sacred and profane, the hierophany coincides with the symbol that interprets the role of an 
instrument in charge of expressing a revealing concept that, in religion, transcends from the human di- 
mension to flow into the myth that connects man (fragile and corruptible) with the world of divine per- 
fection: that supernatural universe so excellently represented in the nature proper of legends and myths. 

These arguments offered a great variety of choices. An example for all is the struggle between a su- 
perior and an inferior principle (divine and animal world) personified by “Mars tamed by Venus” or “the 
monsters” encountered by Hercules during his fatigues. Both expressed the continuous tension of the 
human soul suspended between virtues and vices: man, tending towards good, was incapable of aspiring 
to perfection and was often threatened by the danger of falling back towards the irrationality dictated by 
the instinct. 

Here it is not possible to exhaustively discuss the examination of the potentiality of the method and 
the synergies that can arise from its constant implementation. 

Therefore, below, we will limit ourselves to illustrate two examples that explicit how it is possible to 
make the research of comparative meanings between apparently unconnected cultures, emerge. 


Iconographic convergences. The Mandala and the hyperuranion: micro and macro 
cosmos, meditation and transcendence 

The geometric forms of the mandala, the cosmograms, represent the passages and the respective 
rituals of the meditation path as well as the testimony of man's ability to crush evil and reach enlighten- 
ment and eternal salvation: concepts also reaffirmed in the Yamantaka depicted in the act of crushing a 
buffalo under his feet, which interprets the ambivalent identity of destroyer and destroyed. 

The Mandala 

In Sanskrit the word “mandala” means circle or cycle. 

Originally the meaning coincided with that which possesses the essence (la) and the totality (manda). 

The mandala is a diagram that shows the structure (and unity) of micro and macrocosm of the inner 
and external world. Its configuration represents the path to achieve illumination. 

The basic structure is shaped as a temple (or palace) with a door on each of the four sides. The tran- 
scendental inhabitants appear at the centre of the composition and each detail occupies a space respect- 
ing a precise ritual, which translates into a meditation exercise. 

The hyperuranion 

According to the Neoplatonists, the world was organized in concentric spheres, whose extremes were 
Hyperuranion intended as the divine world and matter, understood as an animal world. 

One of the most exhaustive artistic interpretations of these concepts is in the illustration of the Earth 
at the centre of the Creation in Martin Luther's Bible dated1545. 

Thanks to the philosophical speculation, the most noble and elected spirits can experience happiness 
and reach the knowledge of the true after death. Man, endowed with reason, can choose whether to rise 
to the divine world or descend to the animal world, or even keep himself in an equidistant balance. This 
choice is accomplished through the mediation of love and beauty. 
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If we compare the iconographic interpretation proper of the mandala with the engraving that repres- 
ents the hyperuranian in Luther's Bible, we can realize how the two exegeses reveal and highlight irre- 
futable coincidences in the collocation of the figure of the man interior within the cosmos. 

In fact, hyperuranian is that metaphysical dimension merely spiritual described by Plato. 

In this place, without space or time, the immutable and perfect ideas reside and they can only be 
reached by the intellect and precluded to all that is earthly and corruptible. 

Even in the Mandala the transcendent inhabitants, engaged in meditation, search for the way that will 
lead them to divine enlightenment and salvation. 

These passages, although subordinated to various reinterpretations, explicit the same concept with a 
certain coherence: the mystical experience is communion with the mystery beyond any visible or sensible 
form. It is differentiated and subordinated not only according to the social and cultural environment 
from which it draws its origins but, above all, according to the metaphysical and philosophical influence 
proper to the context in which it is generated. 

The Supreme Lord, Saviour of Heaven and Creator of all things, will therefore open the doors to 
Paradise rather than to Nirvana; to the Indian Brahman (source of mystic salvation and explanation of 
the world) rather than to the Shang-ti of the classical chinese sources or to the Ahura Mazda of the Irani- 
an Hvarenah, where its power and its splendor symbolize the nature of the Mazdean cult. 

Even the Islamic religion, apparently far from the concepts of religion that unite humanity towards 
the sharing of eternal illumination, bases its proper belief on the same presuppositions. The latter will 
distance itself from the usual observance of the religious traditions in the rest of the world because of the 
modalities of interpreting loyalty to the Koran and to the Prophet Muhammad: namely to intend how the 
State of Medina should represent the absolute model of every Islamic state to which one should submit. 
This concept also represents the knot that explicits the conflict between traditionalist Muslims or fun- 
damentalists and reformist or secular Muslims. 


The Earth at the centre of Creation 
in an illustration of the Luther s Bible 1545 Tibetan Mandala 
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The role of light in religions: parallelisms between oriental and western iconography. 

The importance of light as a divine source represents the main iconographic link between different 
cultures and interprets the representation of the supreme entity. 

Over the centuries, references to the image of God have been shaped according to a multiplicity of 
socio-cultural “rules”, however the presence of light as a divine source has remained the “ontological” 
principle by antonomasia. 

In the East the anthropomorphic image of Buddha appears around the 1st century AD in northern In- 
dia and its rich iconography refers predominantly to spiritual enlightenment that will guide the devoted 
in his path of faith. 

The interpretation of the divine in the source of light, then, is shared in the mind of every human be- 
ing, shaping itself in the specific context that will personalize the vision subordinating it to a specific 
supreme entity. 

Just think that of the halos of the saints, which are found in the pre-Christian depictions of the gods 
of the Olympus, in the Hellenistic funerary stele as a deference to the world of the afterlife and that were 
handed down by the Buddhist empire Kushan, to Mithraism, to the pharaohs: a common reference to 
numerous religious phenomena. The halo reproduces the image of the sun as a source of light, of life 
and as a symbol of the achievement of eternal salvation and of the truth thanks to meditation and faith. It 
also is reflected in the celestial phenomena, in the metaphysics of light (the first form of material reality 
that constitutes the essence of everything), in the zodiacal signs and much more. 


è 
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Detail Vajrabhairava Mandala 
Yuan dynasty (1271-1368)Metropolitan Saint Michael and the dragon 
Museum New York Raffaello Sanzio, Louvre Paris 


Divine illumination, meditation and overcoming the fear of death from the East to the 
West 
The Buddha Yamantaka and St. Michael who crushes the devil. 
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The characteristic iconography of Yamantaka portrays him with the head of a buffalo, symbol of Yama, 
death. The antaka, instead, means “exterminator of death” (an common aspect to all the Buddhas, since 
they interrupted the cycle of rebirth: samsara). 

Yamantaka interprets the awakening from death, the achievement of salvation and immortality, to which 
one can arrive only after having followed the path of meditation and wisdom through the journey of the 
Mahayana practitioner, to the point of reaching enlightenment. 

Yamantaka symbolizes the overcoming of fear for death because this figure simultaneously imperson- 
ates death and rebirth (the eternal salvation). It ends up by "exorcising" the perception of fear and allows 
it to be metabolized as an intrinsic and concrete existence generated exclusively by the “ignorance” of 
earthly conventions: enlightenment destroys death. 

The sword, in fact, the customary attribute of the Yamantaka, alludes to the defeat of ignorance. 

In Western art the legends, mostly medieval, which tell the stories of the saints sauroctoni (murderers 
of scary dragons), are very frequent. Just to mention, first of all, St. George, but also Teodoro, Silvestro, 
Margherita, Marta (who tamed him). The legend of the Archangel Michael who fought against the apo- 
calyptic dragon was also very flourishing. 


Vajrabrabhairava Mandala Detail Paolo Uccello 

Yuan Dynasty (1271-1368) Vajrabhairava Mandala Saint George and the dragon, 1464 about. 
New York, Metropolitan Museum National Gallery London 

The hair of Buddha 


Some coincidences, influences and iconographic are evidences in the representation of manufactured 
goods in Eastern and Western art. 

An example of philosophical and artistic parallelism is expressed in the comparison between the 
image of Buddha and a bust of a Roman general in porphyry, of the 2nd century AD. kept at the Borghe- 
se Gallery in Rome. 

The ushnisha and the urn are the most important attributes (the laksana) that refer to the spiritual per- 
fection of the Buddha as a cosmic man (mahapurusa) and are those protuberances on the head and 
between the eyebrows. 

The image of this Roman general, who has his hair gathered in an oriental hairstyle, alludes to the 
value of the Roman leader and it exalts the virtues of supreme wisdom. 
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Comparisons between the image of Buddha and the bust of a Roman general of the 2nd century AD kept at the 
Galleria Borghese. 


We thank professor Marco Bussagli, teacher enrolled in the graded list of the first band at the Academy of Fine 
Arts in Rome for the validation of hypotheses and the rereading of the text. 
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SOME REFLECTIONS € QUESTIONS ON EMERGING FORMS 
OF DIGITAL PHOTO-LIBRARIES 


James R Hemsley 
Birkbeck College, London, U.K. 
jrhemsley6@gmail.com 


I would like to reflect here on the hyper-growth of some new forms of photo-libraries: 
microstock photo agencies with global reach and innovative business models, in which 
‘micro’ refers to the amount of money paid to photographer contributors for each electronic 
photo by these ‘middlemen’. This means cents rather than the hundreds or even thousands of 
dollars photo rate which an established photographer can charge: 


‘-- a way that amateur and professional photographers can sell their photos, earning a 
(very small) commission from each photo sold. Millions of photos are licensed for use via 
microstock, some of the biggest names in the stock photography business operate their own 
microstock photo libraries,’ [1] 


Microstock photography is often termed micropayment photography. Some microstock- 
photographers make considerable money using microstock agencies, e.g. Yuri Arcurs. Many 
others aspire to this happy state: for example, Shutterstock reports over 100,000 contributors. 
Photographers often use more than one microstock agency, but the business models are often 
criticised due to the low payments. It is highly competitive. 


The microstock business is also extremely competitive at the inter- company level as well 
as the intra-company (photographer) one. Considering the former now, this is noticeable from 
their respective on-line profiles aimed at both their markets [e.g. advertising] and their 
photographer suppliers. The field has consolidated considerably despite its youth and 
attracted interest from very large companies from inside and outside the photo library fields. 
For example, Getty Images acquired iStock, a top microstock agency, in 2006. Even more 
striking was the takeover of Fotolia by Adobe at a reported price of $800 million [2]. Other 
current leaders include Alamy and Shutterstock. It is instructive to consider the historical 
situation in 2007 [3] and the changes since then. Now some questions 


Could the rapid progress of the sector - including continuing tough Darwinian struggles 
between the top stock photo and microstock agencies (whether they stay separate business 
(sub) sectors or continue to merge) and the rise of new ones - affect the remaining 
independent European heritage photo-libraries, including small privately-owned ones? The 
situation may remind some of us of the fierce debate in the mid-1990s regarding the 
apparently imminent monopolisation of electronic rights over many of European cultural 
heritage top images by Microsoft/Corbis, IBM et al. At that time another emerging threat 
seemed palpable: giant Japanese companies, such as Hitachi and Toppan Printing, acquired 
valuable arts image rights from top European museums and galleries including the Uffizi; 
Japanese admiration of the Impressionists as well as the Renaissance fuelled their efforts. The 
perceived US and Japanese threats informed and stimulated the creation and development of 
Europeana, providing a valuable portal for numerous European Heritage institutions which is 
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still blossoming today (currently just over 50 million records) facilitated by the European 
Commission.[4]. Perceptions of these commercial Japanese and US challenges faded and a 
good case can be made that the Japanese interest was positive for both sides, e.g. between 
Italy, Florence, the Uffizi, and Japan. Are there lessons which may be relevant and useful for 
Europe regarding the possibility of future Chinese entry into this scene; ie. could large 
Chinese tech companies such as Alibaba and Tencent [5] lead a new charge from Asia by 
acquiring European cultural rights? Might large Indian hi-techs do the same? Should 
Europeana, as well as leading European cultural and historical photo-libraries such as Alinari, 
be seen as major cultural heritage bulwarks against such new forms of image libraries from 
outside Europe and North America? Could this perhaps be the case regarding Social History 
(as well as Fine Arts) looking forward two, three and more generations? “Early Twenty-First 
Century” collections may then well seem a valuable part of human History — as *1968” and 
the “Swinging Sixties” seem to many of us now. Could there be constructive, 1f sometimes 
tense, co-habitation and even fruitful cooperation? 


A related topic of considerable interest is the aesthetics of art photography. A current major 
exhibition at the National Portrait Gallery in London on the ‘Victorian Giants: The Birth of 
Art Photography’ in the mid- and late 19" century provides an interesting view of this [6]. In 
this regard Stocksy United Photography, a recent start-up by Bruce Livingstone (founder of 
iStock), is interesting since their claimed USPs (Unique Selling Points) include: ‘curating’ 
their ‘authentic’ offerings, a co-operative ownership structure and a ‘fair pay’ philosophy to a 
relatively limited number of photographer artist partners [7]. Do the existing agencies contest 
this view? Their own quality controls appear to be increasingly rigorous. Will ‘microstock 
agencies’ continue to be (largely) demarcated from ‘stock agencies’? In any case, the so- 
called ‘photo wars’ appear likely to continue. 


Let us now briefly explore the broader context: the hypergrowth of the electronic-based 
Image Culture continues as indicated by the emergence of electronic image-based enterprises 
such as Flickr, Instagram and Google Photos with hundreds of millions of images --- even 
billions. Moreover, we are now no longer surprised by the apparently inexorable global 
growth of the Internet, smart phones with cameras, social media and the corresponding 
FANG behemoths (Facebook, Amazon, Netflix, and Google/Alphabet.) joining Apple, IBM, 
Microsoft and others in the top Technology business ranks with the Chinese, led by Tencent 
and Alibaba increasingly present [5]. However, some observers have even deeper and broader 
concerns regarding some of the latest trends. Sir Tim Berners Lee, the ‘Father of the Web’ 
while working at the European CERN, gave cause for thoughtful reflection recently on the 
Web’s 29" Birthday [8]. He pointed out that ‘over half of the world’s population is now 
connected’ but that nonetheless, as well as grounds for celebration, should be regarded with 
cautious: in addition to the well-marketed benefits the results include increasing threats. 
Another example of a perceived threat is in the field of public broadcasting: even large 
national European Public Service Broadcasters such as the BBC [and presumably RAI] are 
now being increasingly challenged by digitally streamed films from Netflix and Amazon, 
with very significant content production budgets, coupled with the trend of increasing 
numbers of [especially young] people to get their news from their smartphones and 
computers rather than newspapers, magazines and TV sets. At the time of writing, the 
Facebook and Cambridge Analytica furore arouses great concern. Could the area of 
contemporary photo-libraries be affected by such concerns? 
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It is hoped that this short paper will help to stimulate debate and research studies on the 
intriguing topic of new photo-sources and the vibrant scene of photography. 
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Contemporary Art is evolving on different lines. Among many other ones, two important ones are: 
Classic Art and Abstract Art. 

In the Classic Art line, of particular interest is the Renaissance style, where the Artist is 

taking inspiration by Renaissance Art or following some connected guidelines in the 

scenes and Human Figures representation. 

Our Art Production is following this line. 


The Classic Art connection in our Art-Works is based on some main following 
characteristics: 


- represented scenarios (landscapes, Human Figures); 
- structure of the Paintings (borders, geometry, etc.); 
- colours used in the Paintings; 

- harmony of the composition; 

- perspective properties. 


We have painted many contemporary Art-Works and organized several Art Exhibitions on 
religious topics and other ones, such as connected to music, dance, theater. 


The same inspiration was impressed in the many Banners (Palios), now 70, we realized for many 
Municipalities in Italy to celebrate their History Origin and Evolution [1]. 


Our activities are also covering industrial applications, in particular with creative design and paintings 


for the fashion area. 


[1] P. Imposimato "A Cultural Tradition in Italian Towns: The Banners for Yearly Historical 
Reenactments", Proceedings of EVA 2017 FLORENCE, Firenze University Press, Florence, p. 
125, May 2017. 
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Creative perception, decoding of hidden images, 


contextualisation and interactive learning process 


Jeanette Zwingenberger 


Université Paris 1 Panthéon-Sorbonne 


zberg@hotmail.com 


| propose a new approach to art history applying multimedia. My proposal is focused on 
the Piazza della Signoria as an example of cultural heritage and as a tool for research and 
communication. How can we create meaningful participation within the city, discover the 
Piazza della Signoria, learn about history, and share the experience? The aim is to 
decipher the public space and to learn and see it by opening the traditional relation 
between object and spectator through interactive involvement by finding the hidden image, 


the secret message, and understand the story behind it. 


The Enigma of the Piazza della Signoria in Florence 

The Renaissance sculpture displayed in this public space form an impressive outdoor 
museum commissioned by either the Republican or the Medici families. Each of these 
figures is a tool for propaganda. The tour reveals the Piazza della Signoria in Florence as 
the theatre of a political and erotic stage. The actors are the monumental sculptures: 
Judith and Holophernes by Donatello, David by Michelangelo, the Fountain of Neptune by 
Ammannati, Hercules and Cacus by Bandinelli, Perseus by Cellini, and the Rape of the 
Sabines Women by Giambologna. Theses sculptures are interacting like in a performance 
where each artist tries to outdo the others. Every figure of this forum incorporates a 
narrative dimension bringing together several temporalities involving representations of the 
Old Testament, heroes of ancient Greco-Roman mythology, and the mortals: the 
sovereigns of Florence. 

My aim is to analyze the positioning of these sculptures and their interactions within this 
historical space: the Piazza della Signoria, the platform of political and civic life in the city. 
In the 16^ century, the L-shaped piazza was enlarged by the Palazzo della Signoria and 
the Palazzo degli Uffizi within a perspective where the sculptures serve as both guardians 
and focal points: each gaze interacting with one another. 


There is a specific point of view exposing theses protagonists through a totally different 
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context. An ambiguous sensuality characterizes the colossal David by Michelangelo in the 
courtyard’s midpoint as well as Cellini’s Perseus beheading the Medusa, which dominates 
the public area from the Loggia dei Lanzi. The figures seem to be petrified by Medusa. | 


will reveal how through a specific angle they conjure their destiny. 
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Abstract - The Archivision Research Library is a collection of 100,000 digital images of art 
and architecture professionally photographed by a trained architect. It documents the built 
environment--from ancient monuments to cutting-edge contemporary constructions--with 
extensive, standardized descriptive metadata. Archivision is accessible for research and 
teaching through a web-based application--a dedicated hosted instance of MDID--with 
vrcHost LLC delivering full services and technical support: installation, integration, and 
maintenance. This combination provides not only instant access to Archivision, but also to 
sophisticated tools for managing images using an open source media management system to 
discover, aggregate, study, and present digital media. 


INTRODUCTION 

In today’s technology-filled world, we are surrounded by art and architecture, 
often spontaneously snapping and sharing the images that capture our attention. Yet, 
seeing the built environment through the eye of an architect is a singular experience. The 
Archivision Research Library of 100,000 digital images does exactly this [1], with the 
added bonus of also providing extensive descriptive information and seamless hosted 
access in the Madison Digital Image Database [2]. MDID provides sophisticated tools for 
managing this inspiring vision of the built environment using a freely distributed, open 
source media management system to discover, aggregate, study, and present digital 
media, while vrcHost LLC delivers full hosted services and technical support [3]. This 
summary explores the depth and breadth of Archivision and demonstrates the utility of 
accessing the digital images in MDID for research, study, and teaching. 


ARCHIVISION 

The Archivision Research Library contains over a 100,000 digital images of 
world architecture, urban design, gardens, landscapes, archaeological sites, and art in 
museums and public places, professionally shot by Scott Gilchrist, photographer and 
trained architect (Bachelors and Masters Degrees in Architecture) [Figure 1]. The 
collection has been meticulously built over the last 25 years capturing the architectural 
world, from ancient monuments to cutting-edge contemporary constructions. 
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One of the strengths of the library is the completeness of the visual 
documentation of any given architectural work, from overall views to exacting details. 
For example, Archivision holds 295 images of Sagrada Familia in Barcelona, showing 
exterior and interior views as well as fascinating details [Figure 4]. Similarly, for a 
sculpture example, there are 51 images of Cellini’s ‘Perseus with the Head of Medusa” in 
Florence [detail in Figure 5]. What also separates the Archivision material from the 
casual shots many of us take as we travel is the mesmerizing quality of the images, this 
detail of the San Francisco Federal Building by Thom Mayne of Morphosis (102 images) 
and an interior view of the Monastery of Sainte-Marie de la Tourette by Xenakis and Le 
Corbusier (242 images) are good examples [Figure 2]. 

Then, there is the extensive descriptive metadata that is applied to each 
architectural or artistic work. For example, standardized terminology like 
Deconstructivist, government office buildings, precast concrete, galvanized steel, 
perforated steel panels, architecture and city planning, etc. are all used to describe 
Mayne’s Federal Building mentioned above [Figure 3]. The descriptive metadata follows 
the VRA Core data standard for the description of images and works of visual culture [4] 
and Archivision staff can provide clients with this metadata in Microsoft Excel, Core 4 
XML, and RDF with linked open data. For structured terminology, the Getty 
Vocabularies provide authoritative information to insure efficient online search and 
retrieval [5]. Expansive information from resources such as Grove Art Online is added to 
the description field for research and study purposes [6]. 

Archivision is a growing collection with 9,000 new images and descriptions 
added annually. Intended for use primarily in educational contexts, the Archivision 
Research Library can be licensed to own in perpetuity, allowing it to be added to any 
existing digital collection or preservation repository, either by individual modules or in 
its entirety. Or, it can be accessed through subscription or hosted services. Many 
institutions cannot or do not want to load the images and metadata locally, therefore 
Archivision partners with vrcHost LLC to make the collection accessible and usable for 
research, study, and teaching through MDID, a web-based application with great 
functionality (extended information below). In this way, the Archivision content can be 
instantly accessed through a dedicated hosted instance of MDID with vrcHost LLC 
delivering full services and technical support, such as installation, integration, and 
maintenance. MDID provides sophisticated tools for: managing the Archivison collection 
of digital images, studying art and architecture, and presenting a variety of digital media 
in educational contexts. 


MDID and vrcHost 

MDID is a freely distributed, open source web application originally developed at 
James Madison University in Harrisonburg, Virginia, and now supported mainly by 
vrcHost LLC. MDID is a digital media management system with sophisticated tools for 
discovering, aggregating, and presenting digital media in a wide variety of learning 
spaces [Figure 4]. The project started in 1997 in response to increasingly digital 
curriculum requirements within JMU's School of Art and Art History. It evolved over 
time into a cross-disciplinary instructional application used at several hundred institutions 
in the United States and around the world. It is freely available for download from the 
Internet under an open source license. 
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MDID has support for audio and video, flexible metadata structures, a rich and 
robust discovery interface, granular access controls, PowerPoint compatibility, support 
for composite objects, and novel presentation mechanisms that run on most operating 
systems. It ships with a companion application, the MediaViewer, used primarily in 
mediated classrooms to display slideshows (groups of ordered images), featuring intuitive 
zoom and pan controls, intelligent navigation, image caching, catalog data display, and 
support for dual monitors or projectors for high resolution image comparison [Figure 5]. 

Andreas Knab at vrcHost started offering commercial support for locally hosted 
instances of MDID and the hosting of fully managed MDID instances in 2013. This 
service saves institutions from the worries of installing and administering a server and 
application software, allowing users at educational institutions to concentrate on their 
immediate tasks of collection curation, lecture slideshow creation, or research and study. 
The partnership with Archivision pushes this even further by preloading the application 
with a large number of high resolution art and architecture images accompanied by rich 
descriptive metadata. 


CONCLUSION 

European institutions are encouraged to join the over 200 existing clients 
worldwide who are finding the Archivision Research Library to be a key educational 
resource. Hosting in MDID is highly recommended for instant access and the use of 
powerful tools that enhance research, teaching, and the study of art and architecture. This 
collection, curated by a trained architect and professional photographer, provides an 
inspiring vision of the built environment and access to many other associated art works. 
People want to get their information online and this Archivision/vrcHost collaborative 
wants to contribute to the development of digital libraries for current and future learners. 


152 


FIGURES AND TABLES 


Ll 


THE ARCHIVISION RESEARCH LIBRARY 
96,000 professional images with rich, standard metadat 


A H IVI SION for teaching and scholarship in the visual arts. 


Q-— Start Page francais | espe 


THE ARCHIVISION 
DIGITAL RESEARCH LIBRARY 


OVERVIEW OF THE LIBRARY 
BUY 

SUBSCRIBE 

SAMPLE IMAGES + DATA 


HOSTED ACCESS OPTIONS 


for educators and visual resource professionals 


— A R: VA ht 


demo database in LUNA m gardens, sculpture, frescoes, pre- 
landscape rahefs paintings moderr 


Figure 1 - Archivision Research Library Web Site 
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Figure 2 - Sample Archivision Images - San Francisco Federal Building (left) and 
Monastery of Sainte-Marie de la Tourette (right) 
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Date: 
Style/Period: 
Location: 
Location Note: 
GPS: 
Measurements: 
Materials: 
Work Types: 
Description: 


Classification: 
Image Rights: 
Vendor ID: 
MDID Identifier: 
Archivision Set: 
Collection: 
Record created: 
Last modified: 


: San Francisco Federal Building 
: Northwest corner (7th Street and Stevenson St.), viewed from 7th street, short 


end wall of tower with fire escapes and solid panel cladding in irregular shape 
Morphosis (American architectural firm, founded early 1970s); Thom Mayne 
(American architect, born 1944) 

2003-2007 (creation) 

Deconstructivist: Twenty-first century 

site: San Francisco, California, United States 

90 7th Street on the corner of Mission and 7th streets 

+37.779167-122.411944 

605,000 ft2 (area) 

precast concrete; galvanized steel; glass; wood; perforated steel panels 
buildings; public buildings: government office buildings 

Thom Mayne of Morphosis designed the building using a juxtaposition of gray 
concrete walls, perforated metal panels, and custom, faceted wood ceilings, The 
building was designed to be a ‘green’ building consuming less than half the 
power of a standard office tower and was the first Federal Building to be certified 
under LEED criteria (although it only rated silver). It was the first naturally 
ventilated office building on the west coast since the advent of air conditioning. 
However the building has been criticized as being dysfunctional for its 
employees, with inadequate thermal comfort, lighting and acoustics, In addition, 
in an attempt to promote exercise, a skip stop elevator system was designed that 
only stops every 3 floors, requiring able-bodied workers to walk up or down to 
reach their desired floor, This caused most of the workers to use those elevators 
designed for the disabled which stop at every floor, leading to overcrowding. The 
design's nonfunctional use of extended, folded metal sun shading at ground 
level, which in the opinion of some are purely for aesthetic effect, reguired 
extensive galvanized steel bracing, and added millions in materials and 
fabrication costs into the project. (Source: Wikipedia: 
http://en.wikipedia.org/wiki/Main_Page) 

Architecture and City Planning 

© Scott Gilchrist, Archivision, Inc. 

1A1-MAYNE-FBSF-A84 

91427 

Archivision Addition Module Twelve 

Archivision Module 12 

Oct. 15, 2017 

Oct. 15, 2017 


Figure 3 - Archivision Descriptive Metadata for the San Francisco Federal Building 
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Portrait of Antoni Gaudi 


Figure 4 - MDID Works Overview Screen 
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Alternate Title: 


| Creators: y ( View: 
| 


| Date 
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se 


Figure 5 - Image Comparison in the MDID MediaViewer 
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INTRODUCTION 


Archives, libraries, museums, businesses, and individuals have fully embraced the digital era 
and a proliferation of digital assets abound. The challenge is creating, managing, storing, and sharing 
these assets. Metadata is information that helps overcome this challenge. It is data that provides 
definition, descriptions, and context, all set into a structure that describes how the data itself is 
arranged. Metadata helps us understand and organize our digital assets. It can answer questions, such 
as (but not limited to) who, what, where, when and how. Metadata can be used to add value to your 
work by adding standardized information that will aid in structuring, organizing, discovering, and 
sharing your digital assets. 


WHAT IS METADATA? 


Metadata is a structured way of describing an information resource. It can be simple or 
complex, depending on your needs. The metadata discussed in the following paper pertains to 
cultural heritage objects, but metadata can and should be created for anything that can be digitized or 
classified or organized: online catalogs, search engines, books, magazines, newspapers, video, 
music, and so on. 

The following table obtained from the American National Information Standards 
Organization (NISO) summarizes the various types of metadata succinctly. [Table 1] NISO explains 
it thus: Descriptive Metadata is used to find or understand a resource. Technical, Preservation, and 
Rights metadata (all considered “administrative” metadata) are primarily used for the management of 
files. Structural metadata details the relationships of parts of resources to one another, and markup 
languages integrate metadata for structural or semantic features within content. [1] [Figure 1] 

Metadata has an underlying structure to it. There are ELEMENTS (such as title, subject, etc.). 
In the example above, “Creator” has a value of “Green, Ruzzie, United States, American, 1892- 
1956” and the element “Date” has a value of “1941” and so on. STANDARDS, (a group of elements 
approved by professionals) are developed in response to the group need. For example, the Dublin 
Core Metadata Element Set (DCMES) is a schema used to describe digital and physical resources 
that has become a minimum standard in many archives and libraries. [2] Other metadata standards, 
like the Darwin Core, are specifically geared to items that relate to biology and can be viewed as an 
extension of the Dublin Core for biodiversity information. [3] The VRA Core (the standard chosen 
for this example), developed by the Visual Resources Association and hosted by the Library of 
Congress, contains other expanded elements to more adequately describe works of visual culture and 
the images that document them. [4] Those who work with collections of digital assets should choose 
the standard that most closely aligns with the nature of their materials, or customize, if need be. A 
SCHEMA conveys the underlying structure, that combination of elements and standards, the 
relationships between elements, and the rules for use and syntax for the values. The SCHEMA is 
often the markup language that the end users do not see, but allows the metadata to be machine- 
readable. For example, XML and RDF are common markup languages. [5] 
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WHAT IS EMBEDDED METADATA? 


We understand that Metadata helps describe our digital assets. What then, is embedded 
metadata? Embedded metadata is created using a process that converts text into code and adds that 
information about an asset directly into the digital file itself. These metadata bytes are stored just like 
the image bytes are stored. Many of the digital files we use every day already have metadata 
embedded in them. Cameras and scanners automatically embed metadata such as date, time, and 
device settings. Music files, e-mails, texts, web pages, and more are all packed with metadata; the 
“key to functionality of the systems holding the content, enabling users to find items of interest, 
record essential information about them, and share that information with others”. [6] 

Researchers, archivists, curators, and educators often have knowledge and data the go far 
beyond the automatically stored metadata attached to their digital images, but need to find efficient 
ways to add this value. Depending on the nature of the materials and the schema a person wishes to 
follow, there are different types of data that can be added to describe those digital files. This 
additional information about the objects, people, and places described in an image or audio file can 
be embedded. It is important to take the time to decide what information you want to add, and then to 
be consistent in your application. [7] Additionally, administrative metadata can be quite useful to 
manage images, such as telling users who owns the image and how it can be used. This information 
can also be embedded. 

Not only digital images can have embedded metadata inserted; it can be done to digital 
videos, audio files, and PDF”s. Embedding metadata into a file allows the descriptive information to 
travel with the file across file formats and software applications. When you embed data into your 
digital file, it will sit alongside the already automatically generated embedded information, for 
example technical metadata from a camera, and you will be able to search for it and view it on your 
computer. Improving your digital files discoverability and accessibility is a key advantage to 
embedding metadata. Conducting a search on your own computer, in a database, or other online 
collection can typically involve the data embedded in the files. 

Embedded metadata helps reduce the difficulty of finding an image or audio file on your 
computer, no need to recall a lengthy URL or scratch your head trying to remember the original 
source to cite when using or publishing your images. Ensuring that those who might wish to use your 
images are informed about copyright constraints and provided with contact information is easier than 
you might think, since this type of data can be easily embedded. Embedded metadata enables 
functionality that streamlines and enhances your research and presentations by making images, 
video, and audio files searchable and sharable, with easy to identify content, sources, and rights. 


TOOLS FOR EMBEDDEDING METADATA 


There are many tools out there that can be used to embed metadata: Corel Paintshop, Apple 
Photos, ACDSee, Lightroom, Photoshop Elements, and more. They were all developed to work with 
an image caption standard called IPTC, which is the global standards body for the news media 
providing the technical foundation to allow for sharing information about images in a structured, 
consistent way. [8] 

There now are tools available that will allow you to improve workflow, build upon work that 
has already been done, and eliminate the need to slavishly enter information more repetitively, 
saving time and money. Three such tools were developed by the Visual Resources Association, a 
multi-disciplinary organization dedicated to furthering research and education in the field of image 
and media management. [9] These tools were designed to work in Adobe Bridge and can be used to 
create and manage embedded metadata, making your workflow much more efficient. Although most 
Adobe products are now only available by paying for a subscription called Adobe Creative Cloud, 
Bridge can be downloaded for free once you create an account. [10] 
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VRA METADATA PALETTE 


The first tool is a custom XMP palette, which allows you to embed data into your digital 
files. Knowing that those who deal with material culture have a need for more description, 
especially when data about the object depicted is needed rather than just data about the image itself, 
the Visual Resources Association developed a tool that will allow for richer data using IPTC and the 
VRA Core. [11] We call it the VRA metadata palette. This tool allows you to easily enter data into a 
digital file, carefully separating out information about the artwork or object depicted from the image 
data, while also allowing for the addition of administration data (which is useful for archivists and 
librarians, or anyone handling projects with multiple workers and roles). Data can be entered one 
digital file at a time, or in batches when some information holds true for all in the group, such as 
source or copyright. [Figure 2] 

The palette utilizes the Getty Vocabularies Linked Data Service, which includes: the Union 
List of Artist Names, the Thesaurus of Geographic Names, and the Art and Architecture Thesaurus. 
[12] Using such authorities, structured terminology compliant with international standards, helps 
create metadata that is easily searchable and more discoverable. Institutions use metadata standards, 
authorities, and controlled vocabularies for two primary reasons; quality control and interoperability. 
The controlled vocabularies most commonly used for image and cultural object cataloging are those 
provided by the Getty Vocabulary Program. In the example above, the user can search the Getty 
ULAN for the architect “Leon Alberti”, select his name, and the rich metadata (name, nationality, 
role, dates, etc.) are automatically added, along with authority reference ID’s, which are useful for 
preparing your data for use in linked data environments. [Figure 3] 

I’ve used the VRA palette to quickly record data for images taken from books and journals 
on loan. It could just as easily be used by researchers in the field or anywhere order needs to be 
imposed on files before information is lost and identification becomes difficult. Students with 
minimal training can search for names and terms and then copy accurate information into the file. 
The information stays with the file until it is uploaded into a database, added to a website, sent to a 
publisher, or so on. 

Once this detailed data is entered and ordered using the VRA Core standard, the summary 
section will fill this data into IPTC and Dublin Core fields, manually, or automatically (see below). 
Populating the IPTC metadata allows for almost all systems to discover and display the embedded 
metadata. [Figure 4] 


VRA METADATA EXPORT/IMPORT TOOL 


The second VRA tool allows for importing or exporting digital file data to and from 
spreadsheets (perfect for all those digital humanities projects). Using the VRA Export/Import tool 
allows the user to export all of the data from single files or groups of files into spreadsheets. These 
spreadsheets can be manipulated and put into data sets, used to populate databases, and so on. By the 
way, the reverse is true, you can also take spreadsheets of data, import them, and embed thousands of 
files in one fell swoop. I’ve used the import/export tool many times. When I purchased high quality 
images from Archivision, they came with a spreadsheet of rich and complete descriptive metadata. I 
embedded that data into the images using the Import tool before I loaded them into my online 
database. Now when a user downloads one of the images, the data travels with the image. This 
makes it easier for my patrons to search for specific images they have downloaded, as well as keep 
track of the detailed information about each file, with no effort on their part. Here is another 
example; I scanned over 9,000 slides from Binghamton University Theatre Department productions. 
Using a spreadsheet they had produced with information about each production, I was able to import 
and embed that information into all the files in just minutes. Then, using the VRA palette, we added 
copyright and contact information, as well as standardized subject terms. Next, we exported the 
improved data into a spreadsheet that we used to populate data in the JStor Forum, where the images 
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were then stored and published. [13] 

The beauty of these tools can be found in their back-and-forth utility and flexibility: creating 
data, exporting data, importing data, updating data, and more. This gives you the freedom to work in 
whatever sequence or on whatever platform you prefer and, it allows you the option to change your 
mind down the road. 


VRA METASHOTPPT 


The last VRA tool is called the MetaShotPpt. It quickly allows for the creation of Powerpoint 
presentations in which digital images and embedded metadata automatically load into the slides. 
Simply open the tool, select a folder of embedded images, select desired options, and then click 
“Create PowerPoint”. It just takes moments. Nothing has to be retyped, saving time and ensuring 
accuracy. [Figure 5] 


CONCLUSION 


Embedded metadata is so useful, and the time and effort it takes to create and embed data 
yields returns to the end user—saving time, increasing accuracy, and providing long term 
accessibility. The VRA tools are easy to use and free to download from 
http://metadatadeluxe.pbworks.com/w/page/20792238/FrontPage, along with guides for embedded 
metadata and easy to follow YouTube tutorials. [14] 

All of this is made possible because of the power of embedded metadata, which makes digital 
images independent, self-defining, sharable, flexible and portable. 


FIGURES AND TABLES 


Title 
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Subject 

Genre 

Publication date 

File type 

File size 

Creation date/time 

| .| Compression scheme 
Preservation metadata | Checksum 

Preservation event 


| Descriptive metadata Discovery 
Display 
Interoperability 


Technical metadata Interoperability 
Digital object management 


Preservation 


Interoperability 

Digital object management 
Preservation 
Interoperability 

Digital object management 


| Rights metadata Copyright status 
License terms 


Rights holder 


[Structural metadata | Sequence 
Place in hierarch 
| Markup languages Paragraph 


Navigation 


Navigation 
Interoperability 


Table 1. Table obtained from the American National Information Standards Organization (NISO) 
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Item Details 


@ Institution Collections 


Creator 


Green, Ruzzie, United States, 
American, 1892-1956 


Culture 
American 


Title 
Colgate Palmolive Ad 


> | [1941 
"nm AI 


MS 
Figure 1. Example of an image and associated metadata elements 


WORK 
Agent Francesco Talenti (Italian architect, ca. 1300-1370); Giovanni Rucellai (Ita 
Title Santa Maria Novella 
o Date | 1458-1470 (facade) (alteration); ca. 1246-1399 (creation) 
. Style/Period | Medieval; Renaissance 


Cultural Context 
Work Type 
Material 
Technique 
Measurements 
Location 
Description 
Subject 
| Inscription — 
State/Edition 
Relation 
Text Reference 


Source 


Italian 
architecture 
marble 


construction (assembling) 


The second design commissioned by Giovanni Rucellai, the spectacular gr 


buildings; religious buildings; churches 


Grove Art Online; http://www.groveart.com/ 


Figure 2. Screenshot view of Adobe Bridge with VRA palette open 
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WUGAGU vua 


- r- Work Agent (indexed) 


Name Francesco Talenti (Italian architect and sculptor, born 1: - attribution - 
ULAN ID 500116279 


Name Leon Battista Alberti (Italian artist, architect, and theoris - attribution - 
+ ULAN ID 500002025 


+ 


Fill Display always auto fill Display 
architect: Francesco Talenti (Italian architect and sculptor, born 1300-1310, died after 1369) 


Figure 3. Screenshot of the VRA palette showing the links to the controlled vocabularies 


— VRA Metadata Palette (version 201 
r— IPTC SUMMARY a p 


Fill from VRA || clear _ Always auto fill | | Append 


Title Santa Maria Novella 


Headline Francesco Talenti (Italian architect, ca. 1300-1370); Giovanni 
Rucellai (Italian patron, 1475-1525); Leon Battista Alberti (Italian 
architect, 1404-1472): Santa Maria Novella 


Description Francesco Talenti (Italian architect, ca. 1300-1370); Giovanni 


Rucellai (Italian patron, 1475-1525); Leon Battista Alberti (Italian 
architect, 1404-1472); Santa Maria Novella; 1458-1470 (facade) 
(alteration); ca. 1246-1399 (creation); marble; Florence, Tuscany, 
Italy; 12 


Keywords architectural exteriors; Medieval; Renaissance; Italian; buildings; 
religious buildings; churches; construction (assembling) 


Figure 4. Screenshot of populating IPTC from VRA Core 
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eoe MetaShotPpt 


FÉMetaShotPpt UL ov 


association 
FOUNDATION 


JUsers/mfocht/Desktop/EVA Florence 2018/Lawson Mariinsky Select Folder 


9 images to be imported 


Metadata Options Layout Options 
Include Field Label > Image above Text 
© Image beside Text 

IPTC Photo 
Creator 
Title Background Color O 

J Date Created Font| calibri $) 14 [Ej Font Color | 
Location 

Description 
Keywords 


Preview 
Copyright Notice 
VRA Core Work 


Agent 

Title 

Date Created 
Style/Period 
Culture 

Work Type 
Material 
Technique 
Measurements 
Location 


IPTC Artwork 


Figure 5. Screenshot of MetaShotPpt Tool 
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Abstract — Since its opening, the Museum of the New Towns, housed in the Palazzo di 
Arnolfo in San Giovanni Valdarno, has dedicated a particular attention to the relationship with 
its audiences. In this context, the video “Il piccolo Masaccio e le Terre Nuove” has the purpose 
of bringing children and young people, in particular, closer to the museum main themes. The 
video presents a series of very different techniques, such as live shots, taken also by drone, 
Computer Graphics, 2D drawings executed with a tablet, drawings sketched with traditional 
techniques, such as India ink and watercolours, and digital videos taken from Google Earth. 


INTRODUCTION 


The Museum of the New Towns, housed in the Palazzo di Arnolfo in San Giovanni 
Valdarno (Museo delle Terre Nuove - http://www.museoterrenuove.it/), traces the genesis 
and development of the phenomenon of the New Towns, new settlements founded between 
the Thirteenth and Fourteenth century in strategic areas of the territory and under the control 
of a central authority. This phenomenon, widespread in several European areas, is well 
visible also in the Florentine territory with, for example, the towns of Firenzuola, 
Castelfranco, Scarperia, Terranuova Bracciolini and San Giovanni Valdarno. The visiting 
path through the rooms of the museum allows visitors to understand the reasons lying behind 
the birth of the New Towns, investigating the urban and architectural features (which 
revolutionized the spontaneous medieval stratification) and the aspects of daily life. The 
dwellers were migrants from villages, relocated with the promise of tax exemptions, the 
ownership of a building lot and cultivated lands. The museum dedicates specific attention to 
Castel San Giovanni (the early name of San Giovanni Valdarno), whose foundation dates 
back to January 26, 1299, with the aim of ensuring the Florentine power in the area by 
consolidating the road axis along the Arno river and weakening local lordships. San 
Giovanni, which as Castelfranco should have constituted an economic, military and 
demographic center of the upper Valdarno, will succeed in the fourteenth and fifteenth 
centuries in consolidating and establishing itself in the territory as an important urban reality 
and as a commercial hub along the route from Arezzo to Pisa. 
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Since its opening, the museum has dedicated a particular attention to the relationship with its 
audiences in the belief that only an effective dialogue with visitors can improve the life of 
museums. Therefore, multiple work-flows have been activated in relation to the different 
targets - from residents to tourists, from schools to families. In this context the video 
“Masaccio e le Terre Nuove” has the purpose of bringing children and young people, in 
particular, closer to the museum main themes thanks to an innovative approach, capable of 
combining scientific reliability and digital language. 


“IL PICCOLO MASACCIO E LE TERRE NUOVE” 


“Il piccolo Masaccio e le Terre Nuove” is a short animated educational video explaining 
the origins and the history of San Giovanni Valdarno, a city of foundation [1]. 
The protagonist is one of its illustrious citizens: Masaccio, the painter (Castel San Giovanni, 
1401 — Rome, 1428) (Figure 1). He is presented as a kid of the early '400, attentive listener to 
inspiring explanations about the city provided by the Vicar of his time, Giovanni di Forese 
Salviati. The Vicar is used as a storyteller, capable of inspiring in the young Masaccio the 
interest for the rational rules that underlie the design of the city of San Giovanni, as well as 
other urban agglomerations born out of a precise planning. The choice to resort to the figure 
of Masaccio (the real name was Tommaso) as a kid is dictated both by his bond with the city 
and by the possibility of being a vehicle of identification for a young audience, to whom the 
video is mainly addressed. 


Lob 


Fig. 1 — A shot from “Il piccolo Masaccio e le Terre Nuove": Tommaso and his younger 
brother Giovanni. A study on lighting. 
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The story, after a brief contextualization in the present time, takes place in 1409, and ends 
with a return to the present, reconnecting San Giovanni Valdarno to other still existing cities 
born from a rational and planned project. 


MIXED MEDIA FOR AN EDUCATIONAL SHORT FILM 


The video presents a series of very different techniques. It makes use of live shots, taken 
also by drone, Computer Graphics, 2D drawings executed with a tablet, drawings sketched 
with traditional techniques, such as India ink and watercolours, and digital videos taken from 
Google Earth. 


Fig. 2 - A reconstruction of San Giovanni Valdarno and its surroundings as it could have 
been at the start of the XVth century. 


The live shots present San Giovanni as it is now. These shots were filmed by hand-held 
camera in the city streets, to highlight some of their most characteristic historical elements, 
and have been integrated with drone shots. The aerial view is the best way to understand the 
geometry of the city and it leads the viewer towards the San Giovanni of the past, which 
appears immediately afterwards, as a virtual reconstruction. 

The digital model reconstructs the city and the landscape surrounding it (Figure 2). For the 
procedural reconstruction of San Giovanni (with CityEngine - 
http://www.esri.com/software/cityengine) the precise measures and proportions coming from 
well documented studies on the city were used and some details were realized with 
photogrammetric techniques (PhotoScan - http://www.agisoft.com/). Apart from these two 
commercial softwares, most of the work has been completed in Blender 
(https://www.blender.org/) and in an Open Source pipeline. 
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In order to highlight a connection with the artistic nature of the character of Masaccio, the 
video also makes use of the contribution of traditional techniques such as drawing and 
watercolour painting, which visually and creatively embellish the film. 

The ending, with the sequence created in Google Earth, brings the audience back to the 
present day by recalling a technological tool of common use. Through this solution, it is 
possible to offer a suggestive and direct evidence of the persistence over time of urban 
structures organized around precise geometries and shapes. 


Mixed media were very effective in order to optimise the production pipeline. The 
reconstruction of the city started from a .DWG file delivered by the administration and 
reproducing the current version of San Giovanni Valdarno. The procedural modelling rules 
were devised using the information given by Puma [4], Bertocci [2] and Bianchini [3] on the 
road axis coming from the .DWG file. The programming rules of procedural modeling fitted 
particularly well the rational spirit underlying the original creation of the city, conceived as a 
set of regular modules distributed in urban space according to a precise logic. 

For the chromatic choices, for the buildings and for street furniture, the inspiration was found 
in the frescoes painted inside the Brancacci chapel, in Santa Maria del Carmine, in Florence 
[5]. In particular the fresco Guarigione dello storpio e resurrezione di Tabita delivered quite 
a few hints about the colour of the facades of the buildings, the typology of windows and 
doors and other details, deeply inspired by the curtain of buildings lying on the background of 
the opera and strongly resembling Masaccio’s hometown. 


The central part of the video is focused on a communicative effort for explaining the 
historical origins of San Giovanni and the architectural rules underlying its urban design. In 
order to improve the effectiveness for an audience of youngsters, we chose a narration relying 
on different techniques according to what we considered as the best way for presenting the 
different concepts. 

For highlighting the so called “matrix of Arnolfo”, that is the geometrical developmental 
patterns coming from Arnolfo di Cambio urban design, a mix of 2D and 3D is used. As a 
matter of fact, all the urban elements in San Giovanni, from the design of the defensive walls 
to the articulation of the four different building lots up to their hierarchy, follow the "ad 
quadratum" proportioning methodology based on a rectangle whose sides are in relation to 
each other, equal to the root of two. This system allows to reconstruct in a simple and direct 
way, starting from a single element in which these laws are codified, the whole design of the 
city [2]. 

The rules of proportionality are shown with animations over a simplified version of the 
procedural 3D model of the city. 


A different choice, but always in line with the logic of mixed media, was made for the 
sequence in which the Vicar illustrates to Tommaso the historical events that characterized 
the city. The Florentine political and socio-economic strategies that had led to the creation of 
Castel San Giovanni are presented using 2D graphics realised, in this case, with watercolor 
and ink drawings (Figure 3). The 2D elements were animated and integrated in Blender with 
a 3D background, simulating a parchment [6]. 
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Ak n ere MEM — 


Fig. 3 - The use of 2D graphics for summarising some of the historical elements of San 
Giovanni Valdarno. 


Given that San Giovanni is a city of foundation, another central theme in the short video is 
that of the ideal cities; a subject crossing over the whole history of urbanisation, from ancient 
times to the present day, passing through the renaissance, regarding cities whose urban 
scheme reflects principles of rationality both scientific or philosophical. 


Fig. 4 - Tommaso imagines some stylization of ideal cities. 
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To better represent this complexity, the topic has been introduced by the sequential use of 
different techniques including: 2D digital stylizations, realistic 3D representations and 
satellite visualizations, such as those coming from Google Earth. The scene down the 
staircase, in particular, when Tommaso elaborates the information received by the Vicar and 
"dreams" sketches of ideal cities, combines the animated 3D model of the character and 
stylized geometries of ideal cities. The latter have been realized stylizing photographs, 
paintings and images of architectural references in order to highlight the geometries that 
originated cities such as Grammichele, Avola or Palmanova, in a way that can hopefully 
impress the mind of a young audience (Figure 4). 


After Masaccio’s reverie, a brief come back to his time closes the video before displaying 
satellite images from Google Earth of some of the cities already presented as geometric 
skeletons. The aim is to underline the importance and persistence over time of urban 
settlements designed as rational systems. The Google Earth sequence shows how many cities 
around the world have retained their original structure and how it is visible even today. 

The choice to use this technique for the final part and to keep it even during end credits 
makes the message more lively. Google Earth is a tool familiar to young people and with an 
effective visualisation style, very useful for highlighting how ideally designed urban 
landscapes actually affect the whole world, from Italy, to France and Germany, but also 
Brazil or India. 


CONCLUSIONS 


The variety of contents presented in "Il piccolo Masaccio e le Terre Nuove" is a starting 
point for further educational applications, which can be developed for different media. 
For example, the rationale laying behind the educational part of the video can be considered 
as a lesson in itself about ideal cities: the case study of San Giovanni Valdarno can be a sort 
of paradigm that, integrated with further explanations about other cities that underwent the 
same kind of development, can clarify this historical process. 
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Abstract - City waterways are a valuable part of our cultural heritage. Over the years the usage has changed from 
business to pleasure. Regent’s Canal, cutting across north central London since 1820, has a rich social and 
industrial history. Much of this history has been and is being captured via photographs. Many of these are being 
lost due to limited museum resources and disparate collections. This paper reports on phase one of a fifteen-month 
exploratory research project. The research aims to explore ways of aiding image capture, selection, storage and 
retrieval. We hope to link with researchers elsewhere, especially in Italy. 


REGENT'S CANAL € CAMDEN BOROUGH CONTEXT 

Canals came relatively late to central London. Canals, for irrigation and water supply, first emerged some 
6,000 years ago in present day Iraq (Mesopotamia). Subsequently, canals were built by many others, notably by 
the Romans. China was also a relatively early mover, including the Grand Canal of China and early 
development of a lock system in AD 983. Venice is justly famous for its iconic canal-centered build (and there 
is even a ‘Little Venice’ at the western end of Regent’s Canal). Milan developed its system in the 12th century 
with Leonardo da Vinci revolutionizing lock technology two centuries later [1] which is widely used today- still 
more than 400 years before Regent's Canal would be built. The Canal du Midi, built from 1666 to 1681 under 
Colbert, minister to Louis XIV, connects Toulouse with the Mediterranean and stretches 240 km with almost a 
hundred locks; this is now a World Heritage site. Today, the country of canals, the Netherlands is especially 
suited to their use due to being small and flat and having a substantial population, industry and commerce. 


Regent’s Canal was the final link between Midlands industry and London, then the world’s largest port: an 
extension of the Grand Union Canal running from Paddington Basin to the Thames at Limehouse Basin. It was 
termed ‘a road without ruts’, since the corresponding road systems were lamentably poor; moreover, the 
existing water route via Oxford was much longer. When the Regent’s Canal was constructed 90% of its route to 
the Thames was across agricultural land. Its hinterland now comprises five central London boroughs: 
Westminster; Camden; Islington; Hackney and Tower Hamlets with a joint population over one million. 
Building the canal began in 1812 at the tail-end of the Napoleonic Wars; arduous planning, negotiations and 
financial capital-raising extended into the construction period which lasted until 1820. A notable milestone was 
reached in 1816 when it was opened to Camden Lock. Important personalities and organizations were involved 
such as the leading architect, John Nash, responsible for Regent’s Park; the Prince Regent [later King George 
IV]; the Houses of Parliament (Special Acts were necessary); local authorities and numerous other stake- 
holders. The work included the creation of all the necessary infrastructure such as bridges, tunnels, towpaths for 
the horses and donkeys, water- pumping stations; locks and even lock-keeper cottages. London’s Canal 
Museum at King’s Cross provides an historical picture -- about the lives of the workers, the cargoes, horses 
and how canals work. Conveniently close to the canal for an informative and enjoyable detour, it is a ‘former 
ice warehouse built in about 1862-3 for Carlo Gatti, the famous ice cream maker, and features the history of 
the ice trade and ice cream as well as the canal’s. [2] 
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The great Canal Era proved to be somewhat short-lived in London. The first commercial Steam Engines had 
already appeared in mines and other industrial applications well before the Regent’s Canal construction work 
began. The canal system was quickly outperformed in speed, engineering and economics. At the beginning of 
Queen Victoria’s long reign in 1837 the canals were already in relative decline although still increasing in 
tonnage carried. The Railway Era had arrived but canals still played a strong supporting role in London until 
well into the 20" Century. 


The Borough of Camden is extremely heterogeneous; it stretches from Bloomsbury’s graceful green squares 
and part of Covent Garden in the south to Hampstead Village high up in the northern end; from ‘posh’ Primrose 
Hill in the west to mixed but gentrifying parts such as Kentish Town in the east adjoining the neighbouring 
borough of Islington. In the centre lies not only lively Camden Town but also one of the most deprived parts of 
Europe: Somers Town bifurcated by Euston Station. Camden borough’s diverse population of just over 300,000 
is multiplied by the floods of commuters, tourists and visitors, the latter especially on weekends and holidays. 


The Regent’s Canal with its immediate hinterland curves like a ribbon through Camden Borough. From the 
London Zoo (shared with Westminster borough) in the west to King’s Cross in the east is only about a mile, but 
it — and its immediate geographical context - contains the vibrant centre: Camden Market (Lock), the largest 
street market in London [3]. Camden Market is now the main pull factor, especially for tourists and visitors 
from abroad and the UK, but there are numerous other cultural and historical places to visit. The northern side 
includes the new University of the Arts (UAL) Central St Martin’s campus, the Roundhouse Theatre, the new 
King’s Place Concert & Arts Centre and the London Canal Museum, the latter just over the borough border in 
Islington. The southern part of the ribbon embraces the British Library, three mainline railway termini (Euston, 
King’s Cross and St. Pancras) and the splendidly revitalised Victorian St Pancras Hotel. On both banks there are 
art murals and striking graffiti on the walls, greenery, stark industrial heritage buildings, modern apartment and 
office blocks with enormous towering cranes. In the next phase our research will extend its geographical 
context westwards to embrace the Borough of Westminster. 


FORTY YEARS AGO: CANAL BOAT & PHOTOGRAPHY EXPERIENCES AND THE 
NEED FOR PRESERVATION 


The personal recollections of Graham Diprose describe his early experiences as a photographer working on 
canal boats and succinctly provide a compelling case for the preservation of photos of the canals: 


‘My own interest in canals dates from over forty years ago, when as an unemployed college leaver with 
a qualification in photography, I became a volunteer assistant coal merchant on the Town Class Large 
Woolwich Narrow Boat ‘Alton’. This was being restored by the Narrow Boat Trust [4], of which at some 
point or another, I ended up as chairman for a year or two. Almost inevitably, these two strands 
combined, particularly when I became an assistant photographer in a studio over Belsize Park 
Underground Station, just north of Camden Town and the Regents Canal. Apart from taking some 
pictures of my own, I also became involved in copying and restoring old Black and White photographs 
of London’s Waterways and those of South East England for two books by Martyn Denney [5], a friend 
from childhood. Sometimes I was copying tiny prints, while on other occasions we were offered a series 
of glass negatives that had never been printed. As word went out among the canal community that we 
were looking for new and unpublished images, soggy shoe boxes would arrive, rescued from some dead 
relative’s decaying garden shed. Many were beyond saving. Some were well fixed or sepia toned for 
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longevity, while others would already be fading to a pale yellow, bronzed with a metallic silver surface, 
or green with mould. 


When we visited local council and waterways orientated museums, curators would often tell similar 
stories of random boxes of prints arriving, sometimes left on the doorstep after hours. Some would 
become among the most iconic and well recognised and republished images of London 's canals, while 
the fate of others often rested on whether unskilled volunteers could document and do any rudimentary 
preservation on these images to ensure their survival to the present day. What had survived, or been lost 
forever, was totally by chance and random in nature. Perhaps this particular example could be repeated 
in many thousands of different historical subjects and topics, but as waterways provided a particularly 
interesting and well covered subject for early Victorian photographers onwards, it provides a 
particularly thought-provoking case in study. 


Probably millions of photos of the canals have been taken over the years. Many of the photos have been lost 
or are not available for public access, which is as true for old prints as it is for modern digital images. Although 
there is substantial interest in the canals, which is being documented, much is being lost. How is even a 
specialised museum curator to decide which photos is are more in need of "saving" when there are so many? 


Images in the form of digital data are already proving to be far more fragile than either Victorian Black and 
White prints or the Colour Dye Images of the 1950”s to 2000”s. Canal and Waterways small museums/archives 
are particularly vulnerable, due to very limited funding and in many cases, poor IT skills. Voluntary assistant 
curators are often in their retirement years. “Born Digital” images are at greater danger of losses than digitised 
silver- based images; in the latter case, the original can still survive and be digitised again. 


Waterways Museums tend to be poorly funded and staffed by unskilled voluntary labour, which means that 
any donation of historic canal pictures on film (transparency or negative) risks being hidden away in a box and 
later discarded to make more space. There is considerable pressure to apply for Lottery and similar Heritage 
Grants in order to digitise both Black and White prints and Colour images. This is most laudable from the point 
of view of sharing their collection online to a wider audience, although it can raise issues over lost revenue, 
where print sales would previously have produced a revenue stream. However, there remains among many 
curators and the public the belief that canal image digitisation “preserves” the images for longer. 


One can liken many of today’s museum curators who store their digital collection on CD, Hard Drive or 
RAID (redundant array of independent disks) storage, to those Museums that carefully archived the wonderful early 
silent movies of the early part of the last Century in thousands of tin film cans; when opened the Acetate Film 
inside had either disintegrated to dust or was otherwise useless. When we consider archiving digital images 
over the next 50 years or longer far more consideration needs to be taken of “Bit Rot”. This could be due to 
non-migratable file formats, data simply disappearing while being stored or even during repeated migration to 
file formats and hardware beyond our present comprehension. 


The historic images presently in archive in our waterways museums are survivors. There have been massive 
random losses of the photographic records of not only industrial and transport systems but also of the people 
and communities that initially fired much of the Victorian Industrial Revolution. Digital images are an even 
more fragile form of image storage, much of which is unlikely to survive the next 50-100 years even in well- 
funded and highly skilled museum environments let alone in small archives and institutions. There is a need for 
a structure and guidelines in relation to preservation, including education on the "dangers" of digital 
preservation. Such guidance should be used by funding bodies as well as museums. 
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PRESERVING PHOTOS FOR MANY GENERATIONS 


Diprose and Seaborne [6] propose archiving vital images and documents as hard copy inkjet prints, not in 
place of digital storage and migration, but rather as an insurance, using established conservation methods and 
acid free paper and pigments. This would, in effect, be a means of sending our most significant artworks, digital 
photographs and documents forward into the 23rd century. The images could then be recovered from the print- 
out with minimal loss using whatever capture or scanning technology may be available in the future. There 
have been some initial attempts at such an insurance policy; below we present three previous long-term 
photograph preservation case studies and one new one. Born digital photographic imaging does present an 
interesting opportunity. We already know from previous research [7] that certain combinations of pigment inks 
in a few digital printers, combined with particular acid free, non-baryte unwhitened rag papers can produce 
digital prints with an expected life of 350 years plus. 


Case 1: Thames - London’s Changing Riverscape. In 1997, Mike Seaborne, Charles Craig and Graham 
Diprose made a continuous photographic panorama of both banks of the Thames from London Bridge five 
miles downstream to Greenwich: a remake of a panorama first photographed in black and white in 1937 for the 
Port of London Authority (PLA). Concerned that the newly created digital TIFF files would not outlast the 1937 
Silver Gelatine original, even entrusted to the Museum of London’s considerable expertise, they convinced the 
PLA that the safest way to ensure that the new digital panorama would survive for their bi-centenary in 2109 
was to make an ink jet printout to match that from 1937 with the same lengths of sections and locations. This 
allows any river location to be viewed simultaneously with the 1937 and 2008 versions placed side-by- side. 


Case 2: River Thames Revisited. In a further project by Diprose and Robins: *...in the footsteps of Henry 
Taunt’ sponsored by English Heritage, the photographers digitally revisited the hundred or so ‘tripod spots’ of 
the first Victorian photographer to document the entire length of the River Thames. Their new digital images 
went into the Archive in Swindon, alongside Taunt’s Silver Gelatine Prints from 1860-1920, but again, to avoid 
the possibility of photographs from 130 years ago outliving their new digital images, the whole project English 
Heritage National Monuments Record (NMR) was archived as A3 digital ink jet prints as well as Tiff files. 


Case 3: John Cass East End Archive. This historic and contemporary photography and arts archive was 
originally planned as a digital only collection. Diprose and Seaborne used the opportunity to explore further 
aspects of ‘sending collections into the future’ such as ease of access using multi-image pages on A2 sheets and 
the risks of losing colour management information during repeated future migrations. 


The corresponding EVA 2014 Florence paper [6] provides the three case details; it attracted lively discussion. 


Case 4: A New Case Study at Oxford: The archives of University College, Oxford University hold precious 
documents and images going back to the Renaissance and before. This case will be presented at EVA London 
2018 [8] by Graham Diprose, Michael George and university archivist Robin Darwell-Smith. 


These four case studies indicate that the very long-term preservation problem is attracting increasing attention. 
We hope for much more debate on the concept, scope and details but emphasize that this approach is ‘an 
insurance policy’ for solely digital methods, i.e. in addition to whatever new innovations will appear, not as a 
substitute. Moreover, it is proposed for just a small selected sub-set of the images of a museum or collection; 
this brings up the ‘selection dilemma’. 
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This paper reports the first steps of a potential fifth case study, or at least a “thought experiment”, which opens 
the possibility of extending the scope of such techniques to the tributaries of museum archives: that is to the 
personal collections of current and future photographers which — if they are of potential historic or artistic 
importance — may sooner or later feed into museum holdings as ‘trusted repositories’ for access. What we hope 
to produce, inter alia, by mid-2019, from this case study of Regent’s Canal is a policy for documenting, 
recording and then preserving a small number of selected images and their contexts for access and use hundreds 
of years into the future. There are numerous approaches to short and medium-term preservation of photos and 
other artworks. We shall explore some of these later in the project. 


FIRST STEPS 


Creating a Thesaurus 

The Camden section of Regents Canal has a long and proud history, its use changing over its two centuries 
from business to pleasure. It has provided a home for waterfowl, fish and humans. Along the towpaths, and in 
the water, there are many plants. Every day many people walk or cycle along the canal or spend their breaks on 
one of the many benches. There are many possible professional users of Regent’s Canal photos, e.g. journalists, 
marketing agents, museum curators and librarians, as well as the general public. 


How can all this diversity be captured and made available for retrieval? How can a record of the life on and 
surrounding a canal be created? One method used by museums to deal with the complexity involved is to create 
a thesaurus. The Collections Trust, a UK internationally-oriented organisation which aids museums to create 
documentation standards employing specialised thesauri as appropriate, defines the term as follows [9]: 


“A thesaurus is a structured wordlist that is used to standardise terminology. It provides the user with a 
single preferred term from a hierarchy that is cross-referenced to other groups of terms which may be 
relevant to the subject. Using standardised terminology like a thesaurus to catalogue objects in your 
museum collections enables you to keep consistent records which give reliable answers when searched.’ 


The main purpose of a thesaurus is to aid indexing, structuring and retrieval of specific objects in a collection 
(or information about these). In order to achieve this the structure of the ‘wordlist’ is key. In creating a 
thesaurus to eventually serve as a guide for creation, selection, sorting and retrieval of Regents Canal images, 
key measurements of success are Ease of Use and Usefulness of Results. If it is not easy to use or (as intuitive 
as possible a Mac designer might say), or if the search results are not useful then it will not be adopted. A search 
result is not useful if it either delivers too many results or too few. A too restrictive hierarchy for example 
would mean that not enough suitable results are delivered, for example an image researcher wanting to find all 
photos of Kings Cross Basin but only seeing those from the year 2015 - correct classification and filtering 
would have avoided this error. Creating a suitable structure needs to be tailored to the subject matter at hand, for 
example, a structure for photos of the life on a canal needs to be different than a structure for photos capturing 
the life of an indigenous tribe. It is likely that a search for photos of a specific subject will be limited by dates, 
area or desired perspective. Ideally all this information will be saved with the image for example in the form of 
keywords, so that retrieval is as easy as possible. 


For the hierarchical structure of a thesaurus in this case a categorization via Content is assumed to be the 
most suitable. For example what do we consider to be the content of life on the Camden section of Regents 
Canal? Is it the wildlife, such as the ducks, fish and swans? Or the humans making use of these water lanes? Or 
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the “infrastructure,” such as locks, tow paths and bridges, which also determine what use can be gained from the 
water? And what other factors play a role? A tentative first selection of categories can be made as follows (see 
Figures 1-7 for examples) 


* Nature - e.g. Flora and Fauna, Weather, Water 
+ Canal Infrastructure - e.g. Locks, Moorings 
+ Humanity & closely related objects and activities, e.g. Narrow boats, Canoes, Graffiti, Walkers 


Some examples of key search related questions still to be tackled are "How to choose a category if a photo 
includes elements of various categories (e.g. Ducks swimming in front of a narrow boat just approaching a 
lock)? How can relevant preservation information best be included? Envisaged next steps include comparing 
the initial thesaurus with museum thesauri on similar subjects as well as the types of structures used by 
historical photo libraries and commercial stock agencies (briefly discussed below). After the first iterations, 
which will probably include more substantial changes, the thesaurus will be enriched with suitable definitions of 
the terms used and the desired scope of the categories. 


Current Photo-library and Microstock Agency Practice. 

We made an initial trial search of one photo library and one microstock agency. Founded in 1964, the Mary 
Evans Picture Library [10], a specialist historical photo library (partnered with Alinari), currently have ‘only’ 
half a million images online, adding about 100 daily, with categories and tagging. The term ‘Regent’s Canal’ 
had 44 hits, all but one being black & white as to be expected from an historical archive. In contrast, Alamy, a 
large agency based in Oxfordshire with a reported 127 million ‘images, vectors and films’, the majority in 
colour, increasing by 100,000 per diem, was founded in 1999 [11]. They also use a category and tagging 
system for classification and retrieval. The search term ‘Regent’s Canal’ returned the high figure of 7,529 stock 
photos and images. Drilling down from top-level categories is required and we shall consider leading current 
practice for photo libraries in the next phase of our research. However, this first comparison shows the much 
greater current image resources available from microstock agencies, but they do not have (yet) the historical 
resonances of, for example, a photo of a narrowboat from a century ago. 


We close by referring to an aim regarding evolving from ‘multi-disciplinarity’ to ‘interdisciplinarity’. For our 
work we define Interdisciplinary Research as: ‘Bringing together different perspectives, experiences and 
backgrounds to enable a more holistic approach’. *Multi-disciplinary' may be regarded just as a set of different 
disciplines. The evolution from one to the other will be observed as the project advances further. We will also 
consider other perspectives: walkers, boat dwellers and office workers, such as this office-based journalist 
admirer of the Canal: 


‘For the past two summers, from windows in the Guardian office in London adjoining Regent’s Canal, I 
have watched a pair of swans raise two flotillas of cygnets. She broods, he patrols, and the young ones 
go from fluffy cuties to scruffy teenagers who stay in line far less during family outings on the dark 
water between the moored barges. It’s a rather peace-inducing flash of nature in the relentless, hard- 
edged crush of this great city.’ [12] 
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Figures 1-7. Examples of Nature, Infrastructure and Humanity on the Regent’s Canal 
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Figure 1. A Hungry Visitor 
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Figure 3. Winter Arrives 


Figure 5. Lock and Building in Progress 
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Figure 6. Camera p Tourists 


Figure 7. Fun on the Canal 


Photographs by Christina Hemsley 
(Copyright, 2018) 
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Abstract: The paper examines several examples of visualization common to the contemporary Russian 
scene, which 1) follow the long tradition of elaborating texts; 2) are developed on the border between 
the science of geography and visual art; 3) belong to the sphere of amateur production that by trial 
and error method propose a special 3D filming as a proper way to present an archive. Among other 
projects concerned with questions of memory work and its visual presentation, it is the latter one that 
orients itself towards the possibilities of the future. 


The technological development knows no boundaries, and in Russia, as well as in other areas, it 
makes the work of translation of cultural heritage in digital format possible. As usual, local context 
sets the direction of how visualization technologies would be applied, what parameters would be 
chosen to structure the process of visualization itself. The following is based on the hypothesis, that 
Russia's context is in many ways determined by the length of extended territory and «the feeling of 
space». Accordingly, visualizations often seem to be guided by imagination categories and have 
map as their principle model. These characteristics form the frame in which IT programmers embed 
their encoding and build visual presentations of their products. 


After the euphoria of the 1990s, the state was to pay attention to cultural heritage. New 
technologies are being used to register the conditions of various objects that need to be preserved, to 
predict the future without any premises, as in the case with Zariadievo park pavilion, but often they 
also reveal for the first time something that was produced dozens of years ago, survived the 
revolution, underwent the first decades of Soviet era and was nearly forgotten after the war. This is 
the case with 1) film restoration and 2) stereoscopic cinema. Both are part of long tradition that has 
an origin in avant-garde movements common to Russia and European counters, and present a 
normal activity within the visual field today. Digital mixing of color-separated negatives, time-lapse 
image restoration of color animated films shot according to Soviet tricolor system, such as «Santa 
Claus and the grey wolf» (Lenfilm studio, 1937), «Winter's tale» (Soyuzmultfilm studio, 1945), 
«Carnival of flowers» (Soyuzdetfilm studio, two-tone system, 1935) and others, are a part of the 
transfer work that uses new technologies to open to the public the archive of visual heritage [1]. 
These films are being presented to public already in the new elaborated form, and it is questionable 
whether anyone except specialists would be able to watch them in «old», almost unrecognizable 
form, since current technologies seem to change the habits of viewing and watching dramatically. 
But this is the question to be asked again on behalf of the last example here. 

Another way to use new technologies of visualization is also rooted in the development of film 
production, this time related to the stereoscopic cinema. An outstanding film director S. Eisenstein 
was a known connoisseur of stereoscopic cinema, he considered it the technology of tomorrow: «.. 
this is much dynamic drama! - the interaction of the foreground and depth, from which it actively 
emerges, breaking the foreground» (1947). He had obviously an experience of watching films of 
that type, since the first stereo cinema was opened in Saint-Petersburg in 1911; the world's first 
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glasses-free stereo cinema was opened in Moscow in 1941. 

The development of a glasses-free system started at Scientific Cinema-Photo Institute with the use 
of a raster screen, proposed by the inventor S. Ivanov, in 1937. The raster stereo screen provided the 
formation of the so-called zones of selective vision of the left and right images of the stereo pair, 
these zones diverged from the center in a horizontal plane at the level of the eyes of the audience. 
First films based on Ivanov-system were done with the placement of a stereo pair in the area of one 
frame of 35mm film. The first film in this system was "The Concert" (director A. N. Andrievsky, 
cameraman D. V. Rusenski), shot in 1940 and shown on a wire raster screen at the Moscow stereo 
cinema in 1941, that is, right before the war. 

Although now the adherents of stereoscopic cinema exist, seem to have their niche and promote 
their art of making cinema, touristic products, exhibition design etc., I traced these examples rather 
to indicate that after avant-garde period they were replaced by visualizations of another type, 
namely those connected with «written word» in general and literature in particular. 


Expanding the text images 


The proportion of the visual in the cultural production, perception, and habits needs to be pointed 
out. On several reasons it's so occurred in Russia by 19" century that generally the word had more 
cultural weight than the image. One could even speculate that historically the visual context was 
changed so violently for so many times, that it finally affected the way of visual perception itself in 
such a way, that it can be given only a subordinate role. Apparently, it is necessary to speak about it 
in the present time, because in some examples it is obvious that it remains so, since the most 
frequent translations into digital format are the translations of written documents and ”great Russian 
literature". 


Therefore, these visualizations are inscribed in what is now called «computational philology». They 
are being made by institutionalized specialists in philology (though it is claimed that linguists, 
philologists, historians, culturologists, computer science specialists and media designers work in 
these laboratories side by side, de facto they unite mainly linguists and philologists). Accordingly, 
their main interest is still work with texts, and these are texts that they «visualize»: no work of 
Russian avant-garde artists, video art or Japan manga, but solely manuscripts. The corresponding 
concept of visualization is quite showing. 


I'd like to demonstrate it on the example of Digital Humanities Center, a small branch of the faculty 
of humanities of Higher School of Economics, Moscow. Organized in 2016, the center declared to 
work at the intersection of computer methods and knowledge of humanities; with intention to pay 
special attention to the practice of digital publication, methods of network analysis and computer 
analysis of literary text, the creation of genre structures and electronic tools for research in 
humanities. 


Their projects include : world map of scripts (the user clicks on certain region on the world map, a 
side window opens and shows details of the language, alphabet and scripts to be loaded if needed); 
Persian poetic corps; a parallel presentation of the different translations of the "Poetics" of Aristotle 
(clicking each section of the text coded in figures opens a file with original version, then 3-4 
translations into Russian made in different periods of time, then English and other versions, 
including Persian; one following the other, loaded in one file, no visualization techniques added ) , 
network analysis of drama and similar others. Their another project is the Easy Linavis Web service 
that allows to quickly turn text or other data into a graph and explore it using network analysis 
methods; programming skills are not required. 


This laboratory took part in another massive project, "Tolstoy in one click", in collaboration with 
Samsung Company that developed special apps, and the State Museum of Tolstoy, guided by one of 
the great-granddaughters of the writer. The level of interaction with the audience was valued as 
extraordinary (in two weeks people of different professions and age : engineers, IT professionals, 
doctors, teachers, geologists, linguists, students and other participants from 49 countries read all 90 
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volumes, or 46,000 pages of the works of L. Tolstoy. At the first stage, the participants had to 
download ABBYY Fine Reader and check all the texts to eliminate possible errors that occurred 
during digitization. The second phase of the project involved a more thorough proofreading of the 
texts. After the third, final proofreading the results were converted to PDF with a text layer, and 
also in e-book formats .fb2 and ePub), while visual side was, again, very simply organized. 
Obviously it was not the main task, to make Tolstoy's texts visual, but only to make them readable 
because downloadable. The main objective of the semantic marking of Tolstoy's electronic 
publication was to reproduce, at a new technological level, metatext information, critical apparatus, 
comments and pointers accompanying Tolstoy's complete works edition. What the authors call 
«interactive graphs of the relationships of the characters of the novels» and «visual analysis» is at 
best a kind of a static map, or better a diagram where lines are drawn between schematic «portraits» 
of characters. 


Before going further let's fix this connection between visualization and a static map that presents 
itself in all above mentioned examples: you see the connections between extracted elements when 
you imagine the novel as a whole, a vast world of all existing languages, or a long treatise. In a way, 
one does not need this type of visualization if she or he reads attentively and knows what the text is 
about. The weight of visualization is set by the fact that its object can be seen on the screen of PC, 
though in its still textual form. It seems to remain a part of a written archive, an archive of 
utterances, rather than a mediated visualized from that new technologies allow producing. 


Nevertheless, there is one more type of examples which on the one hand uses data obtained in 
different ways (not written text only or pictures exclusively, but necessary archive work) and, on 
the other, combines it with map-like presentation. This is the case of Dynamic Gulag Map, being 
done in collaboration with Oxford University, Great Britain. A geographic information site of 
several regional maps showing how the Gulag shaped the landscape in the Soviet Union's 
peripheries and of the countless number of people who came through them. The geography of the 
Gulag was complex and penal institutions were not fixed in time and space, some local places have 
long since disappeared from site, overgrown by forest; some continue to perform the same role in 
the present day. «The distinctive geography of prisons demonstrated on this site goes some way to 
explain why certain elements of past practices still plague the penal system, such as the etap, or 
prisoner transports, that can run into weeks taking up new rights to visit prisoners» [2]. This 
example is boundary also in the sense, that visualization here is neither an entertainment nor 
heuristics or epistemic tool: it literary or better formulated, in conventional visual form of several 
simple maps (and some of them can be applied one onto the other by the viewer) opens the distant 
land for the viewer's imagination. 


Though here, too, a kind of discrepancy between the formulated tasks and final (though changing 
upon new information being collected) visual form is obvious. As philologists intend to «visually 
analyze Tolstoy's roman» and do not show how it could be done in their de facto absent 
visualizations, the organizers of Gulag maps, doing their uneasy project write of «optimal way to 
represent the data» and «coming to new analytical indicators with the help of spatial analysis 
methods», and again, restrict themselves a lot with presentation techniques and visualization 
capacities. 


Between artistic production and scientific research 


A group of complex visualizations appear at the junction of art and science, namely the science of 
geography, the emergence of the latter is not surprising if one bears in mind the mapping tendency 
in visualizations that was already pointed above : S. Gavrilova needs visualization to take the 
distant peninsula Chukotka from the imagery zone and present it as a "Not-mainland" (the flight 
Moscow - Anadyr is the longest in the world over the territory of one country); going in the 
opposite direction a cultural geographer M.Kaluzkov seeks to explain the artistic work by means of 
the peace of the land where a work of art was created. 
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S.Gavrilova, a professional geographer and then an artist, a nominee of Kandinsky Art price (2012), 
spent some years studying various zones of the country: the current state of the territory of the 
Leningrad blockade ring, empty spaces of the main building of Moscow State University, visually 
reopened the territory of New Moscow turned into a big construction site. Her video installation 
"Pits” shows landscapes, built on the scheme of geological samples. Meditative contemplation of 
landscapes, similar to the content of the "tubes", allows focusing on the very horizon, which is often 
hidden in the urban areas, but serves a necessary part of our worldview. Landscape panoramas, 
applied to the mirror surface, offer images to be completed by the viewer, relying only on its visible 
parts. This work literally explores the emotional sensations that arise because of a forced change of 
environment, when the viewer tries to complete the picture by only the fragment of it, sweeping 
away what surrounds her or him at the moment of exhibition. 


All together allows understanding the connections between these landscapes and the viewer of the 
exhibition, and what they are in. According to the artist, thus formalized experience is specific. You 
cannot see «live» those kinds of landscapes that are presented for viewing, because many of these 
areas are rebuilt, some wasteland overgrown, some filled with something else. Time passes, and the 
landscapes change, and those at the exhibition are moments of their existence, which the artist 
visited, whether she was in Karelia, the Urals or the Far East. 


This specific balancing between imagined territories and viewing them «in real time» which 
necessarily adds to these visualizations a tone of simulative construction, seems to be more 
appropriate strategy in the land where population long had to invest more in imagination then in 
memory. On the one hand, people of this land still tend to evade solving the extreme problematic 
questions of politics : revolution 1917, concentration camps, dissolution of Soviet Union etc., on the 
other, they do not seem to expose demur against visual activity linked to these events, such as not 
numerous film representations, posters etc. Probably the answer is that they seem to be more trained 
to be more or less critical when the text is concerned, and are able to tolerate a lot in the visual 
sphere, in the sense that visual is not exactly «seen», it is not a part of everyday experience. One of 
the reasons is that a new proletarian state demanded a «new look» of city streets, new people (no 
bourgeoisie), new visual system. It was necessary to introduce this system as the best in the world. 
For example, the famous Moscow subway was not built simply as transportation system, but as an 
outstanding work of art, a “glee space.” Processes of making visually all new world could be 
described by the formula, taken from the International, a hymn based on a poem by French poet 
Eugene Pettier (1871) : «The whole world of violence we destroy to the ground, and then we have 
ours, we construct a new world ...». This aspect of “destroying to the ground” is to be stressed, as 
this seems to be a guide to understand how the visual sphere is worked through: this is not just 
a rejection of old monuments, this is their destruction; physical, material, cleaning “to the ground,” 
no poetic treatment of ruins, or at least practical use for them; it desired no layers and did not intend 
to keep any. 


Strangely enough the erasure-construction process did not stop, although formally, “Soviet logic” 
should have ceased to exist after 1991. It renews itself in the desire to erect something new while 
sweeping away the old, in the belief that the new is better, more modern, and more functional (these 
three arguments guided the construction of Moscow-City). There seems to be a problem that 
constant erasures do not allow the gaze to turn back in order to see what it slips upon. These 
processes made normal the situation in which it is not monuments that trigger the work of memory, 
but other things — and very often they are private things — which cannot be united into something 
big and standing on the square, like toys played by two or three generations of family members, or 
some crockery pieces, or family photo albums. There seems to be no level of commonly accepted 
visuality. 


Thus the elaboration work seems to be done primarily within the sphere of the visual, whatever 
imagery, and utopian, blind it can be as an objective of cultural heritage preservation. Though Great 
Russian literature covers a big amount of the cultural worldview, thought it cannot do without 
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visual technologies today. Though land is important, it cannot produce its own images, since at 
some places it is still wild and almost undeveloped, at the others it has mixed layers of different 
often destroyed cultures that need to be reinstated in a manner which would be probably a 
supplement to “the self-framing of earth” [3,4 ] and forensic constructions [5]. Therefore a tiny 
and accurate, preferably indirect work seems to be a proper tactics of visualization. That is why 
examples similar to Gavrilova’s seem to be showing the slow graduate elaboration of events and 
their places that not only make look back and remember, but help to watch ahead, and see how they 
can be unwrapped in the visual sphere today. 


«To keep or to throw away»: stereo pairs in a 3D-film 


The last example of visualization would be an amateur one that seems to be more striking then 
professional videos of Gavrilova. Perhaps to some extent unconsciously, it combines characteristics 
that were outlined above: the restoration of stereo images produced almost a century ago that pose 
questions of how we are able to percept a technique which is not commonly used today; the relation 
to the past (not necessarily in the form of memory, but imagery, too), the place of literature, text and 
«pure visuality», an archive, the simulative construction, and the map. Last but not least this is an 
example of a vigorous activity directed by images that finally led to a 3D visualization, an active 
work with the sphere of the visual that is so necessary today. 

In the subtitle are put the exact words of the inheritor of the photo archive of Russian photographer 
S.Chelnokov uttered to explain the main task in a personal talk. The archive consists of more than 
one and a half thousand glass slides and negatives (mainly stereo pairs) and a series of colored 
slides in the technique of autochrome, all made between 1880 and 1917; it also includes Taxiphot 
apparatus for viewing stereo pairs of small format. The collection of autochrome (50 color 
diapositives) is more than the entire collection of Polytech Museum in Moscow, and this fact is very 
showing for the intermediate status of this archive : it is somewhere between private and public, 
between visionary phantasies and documented reality, between garbage and work of art, between 
two countries, between past and future. 


The undecidable status of these images was a riddle for the successor and the inheritor of this 
archive. He was not working with the visual professionally, not at any level. He never photographed 
himself or worked for any institution dealing with images, museum or gallery. Furthermore, he had 
no specific training to be able to watch stereo pair that he inherited. Of course, he has eyes inhabited 
with modern technologies of making/watching images (iPhone apps etc.). He explained he just 
couldn't see them because he couldn't figure out what he was seeing — and that's very 
characteristic. He could not decide what to do with them because it was unclear to him whether they 
were interesting or should they be thrown away. That is why, out of inquietude, he began to look for 
ways to watch them collectively. 


This is how the story of presenting the Chelnokov archive to the public began, at present moment it 
includes: an album of printed photographs where each page contains one photograph converted 
from a stereo pair[ 6], the international conference dedicated to photography in general and 
Chelnokov archive in particular; an exhibition (September 2017-February 2018, Manage Gallery, 
Moscow) that consisted of demonstration of several stereo pairs, a (necessary) map of Russia with 
incorporated stereo pairs indicating places which Chelnokov visited with his camera, and finally a 
3D film produced from stereo pairs. 


The latter is the most successful way to open Chelnokov's photographs to the wide audience, 
because on the technical level it creates the space between two photographs of an each chosen 
stereo pair, and at the same time manages to keep the movement of an eye tracing possible links 
between figures inside the frame. Certainly this combination was not possible at the beginning of 
the 20th century. However, it does not come close to the contemporary model of 3D film either, 
since it does not seem to create a feeling of three-dimensionality in the viewer by kinetics of visual 
images, though it still has music underneath and background voice pronouncing a kind of literature 
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narrative loosely connected to the images seen on the screen. It is also a step aside from the 
cinematic tradition to present a sequence of still photos without travelling («La jetée» (1962) is the 
most famous in this series). Thus the inheritor of Chelnokov’s archive and his colleagues proposed 
a technology of visualization that uses new media differently: on the one hand, apart from 
contemporary usage of technology, on the other hand, creating a supplementary view of old photos. 
This is a way by which a technology allows viewing these photos “for the first time”, 
notwithstanding the fact that they’ve materially existed during almost a hundred of years. 


Of course it is rather a construction, not merely reconstruction; it contains clear signs of «today» 
manifesting the specific media shift, but exactly this opens new perspectives in what was 
considered familiar, what was considered “heritage". This constructive converting of Chelnokov’s 
photo archive into specific 3D film makes these stereo pairs a part of contemporary visual field 
since it is being brought out of oblivion, but in addition this visualization demonstrates how the 
future can be open as a variable, how the work with the visual by visual technologies can be done. 
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Abstract — This paper presents the first results of HERIBITS, a research project co-funded by the Tusca- 
ny Region Government, under the Operative Regional Program FESR POR 2014-2020. The project pro- 
poses an innovative methodology as well as advanced technology platform enabling a new paradigm for 
bottom-up and top-down management of cultural heritage initiatives. The techno-social platform integra- 
tes collaborative tools for CrowdSourcing, as well as analysis tools for rating project ideas and evaluate 
socio-economic impact, to propose best practices and to detect similar initiatives in order to avoid project 
duplications. The platform provides also social network capabilities and integrates an ad-hoc CrowdFun- 
ding shop. 


INTRODUCTION 


Cultural initiatives such as the protection and enhancement of cultural heritage, the promotion of local 
identities and the impulse to cultural tourism often suffer of a chronic inability to effectively engage 
and spend substantial community and national resources for underdeveloped areas. 


In fact, there is virtually a lack of structured methodologies that provide the correct time to market of 
planned initiatives, as well as the alignment of projects to industry best practices and a coordination of 
cost centers, avoiding duplication and lack of critical mass. 


This paper introduces the results of the research project HERIBITS, co-funded by the industrial re- 
search program “Bando Unico della Ricerca” issued by the Tuscany Region. The project proposes an 
innovative methodology as well as advanced technologies enabling a new paradigm of collective 
awareness in the planning, sourcing and execution of Cultural Heritage initiatives. 


The objective of the project is to prototype a software platform able to support the process of selection, 
design and implementation of a large scale of initiatives of cultural innovation, ensuring: (i) timing; (11) 
sharing from the bottom; (iii) the quality of project management by comparison with certified success 
cases; (ili) the socio-economic effectiveness by effective and objective methods of rating; (iv) sustaina- 
ble integration in the social fabric, through a direct participation of the stakeholders involved; (v) shar- 
ing between regions, adopting proper communication and sharing initiatives. 


To this end, the project will develop techniques to use linear funds to manage the exponential complex- 
ity of the aggregated cultural initiatives: (1) crowdsourcing and sentiment analysis, (11) project analysis 
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and matchmaking, (iii) rating of socio-economic impact and social crowdfunding of project ideas, (iv) 
adapting and refining the existing methodologies, and (v) integrating them in an original way for the 
aims of the project. 


To demonstrate the method, the HERIBITS project is developing an innovative web platform that will 
integrate several application modules to crowdsource a project ideas, to optimize them both technically 
and structurally, to asses their impact, for crowdfunding and stakeholders participation) in a user inter- 
face oriented to usability and characterized by a sharp style of organizational communication. This 
platform can be viewed as a project factory based on sharing models of proven effectiveness, integrat- 
ing crowdsourcing tools as well as assessment techniques to evaluate the socio-economic impact of the 
initiatives to be implemented. 


By implementing matchmaking techniques for candidate initiatives to find relevant success cases, the 
project factory strongly focuses on clustering and re-use of certified existing solutions in contrast to the 
current state of obsolescence, mediocrity and duplication of projects, amplified by the multiplicity of 
centers of public spending. In addition this platform provides a mechanism of project management 
based on social revenue crowdfunding solutions involving circles of "social investors" clearly identi- 
fied as indirect beneficiaries of the innovation projects, in a context of accountability and effective 
communication of the initiatives implemented. 


The most relevant areas of innovation are the deployment and integration of a set of artificial intelli- 
gence technologies (machine learning, neural networks, text analytics, ontologies) to analyse unstruc- 
tured data and find meaningful relationships (i.e. matchmaking of projects ideas and solutions, identify- 
ing new significant evaluation metrics). 


The HERIBITS project is also developing solutions of socio-economic analysis of the cultural projects 
dropping at the project factory, in order to determine their priorities according to their expected returns 
and to their contribution to higher level development policies. In addition, financing management solu- 
tions are adopted based on a crowdfunding engine able to connect the investment projects to stakehold- 
ers who will benefit directly or in terms of visibility. 


METHODOLOGY 


HERIBITS developed a techno-social web platform, able to support shared programming by accepting 
bottom-up requests, and involving the community in the processes supporting the long-term manage- 
ment of the works carried out. In the current panorama of consultancy services carried out in isolated 
(and to some extent intentionally esoteric) environment of the regional programming technostructures, 
HERIBITS proposes a decidedly innovative approach, in opening the process, through appropriate 
tools, to a Collective Awareness methodology. It is a real revolution, which can, even if with some im- 
plicit risk in the methodological novelty, open new markets and effectively change the current rules of 
the game. 


HERIBITS intends to support this transition not only with the "bottom-up" balancing of the traditional 
top-down logic of programming, but also through the introduction and integration of technical modules 
able to direct the change towards new objectives, guaranteeing greater socio-economic efficiency. of 
the shipyards initiatives, their correspondence to criteria of structural solidity and non-duplication, and 
the rooting of the results achieved over time, according to new shared management models. 


We started by reengineering the entire process of conception, formulation and implementation of cul- 
tural innovation projects for territorial development, through a critical examination of concrete cases, 
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and with the support of domain experts. To this aim, we defined a set of key succeed factors as well as 
critical aspects in need of optimization. Such optimization is realized by leveraging the technical mod- 
ules of the HERIBITS platform, capable of giving an effective response to the critical issues that have 
emerged. 


Subsequently, we translated the set of requirements and functional needs emerging from the reengi- 
neering process into a complete system architecture, defining from a technical point of view the differ- 
ent application modules that the system needs, their operational integration and the general characteris- 
tics of the collaborative techno-social platform provided by HERIBITS. (web system with cooperative 
nodes, inspired by platforms of collective awareness). 


Then we realized the technical modules composing the HERIBITS platform, namely: a collaborative 
tools for CrowdSourcing, analysis tools for rating project ideas and evaluate socio-economic impact, 
propose best practices and advice on possible project duplications, as well as an ad hoc CrowdFunding 
shop. 


We implemented the modules composing HERIBITS techno-social platform according to an iterative 
software development process, as foreseen by the incremental and evolutionary software development 
paradigm known as Unified Process (UP). UP is the process for the development of systems based on 
UML (Unified Modeling Language) and object-oriented analysis and design (OOA & D), which fol- 
lows an iterative and incremental approach, guided by use cases and focused on the system architec- 
ture. 


HERIBITS TECHNO SOCIAL PLATFORM 


The HERIBITS platform is composed of the following technical modules: collaborative tools for 
CrowdSourcing, analysis tools for project rating dedicated to the a-priori evaluation of potential 
project outcomes, to evaluate socio-economic impact, propose best practices and de-duplications, 
that is advising on existing similar projects, The platform, enabled with native as well as third party 
social networks capabilities, integrate also an ad-hoc CrowdFunding shop. The key innovative as- 
pects of such modules are detailed in the following paragraphs. 


The module for explicit and implicit crowdsourcing methodologies of cultural innovation is dedi- 
cated to designing, modeling and technically specifying the reporting software module from the bottom 
of design ideas, and of presenting successful cases with storytelling techniques. 


The module for for crowdfunding cultural innovation initiatives is dedicated to implementing the 
most suitable crowdfunding models and methodologies to maximize the involvement of local stake- 
holders in the launching and running of new structures and services of cultural and tourist interest. 


The module for the deduplication and best practicing of project initiatives is a design optimization 
software module which performs functions of deduplication (analysis of similarities and reconditioning 
of polygenetic instances to unit models implemented in a shared way) and best-practicing functions, 1e 
analysis the weaknesses of each project compared to the best practices certified, with suggestions for 
the actions needed to optimize the organizational structure and operational parameters. 


To this aim, we defined deduplication and best-practicing analysis models for design optimization, and 
seamlessly integrated into HERIBITS platform the third party analysis tool Asset from K4D 
(Knowledge for Development, a socio-economic analysis consultancy dedicated to public administra- 
tions). 
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The study of a methodology aimed at defining the criteria of deduplication and best practicing was 
based on the analysis of questionnaires submitted to project designers and/or project partners of 25 
tourism and cultural heritage projects. Such projects have been selected in different Italian regions (ge- 
ographical criterion) and for different types of actions implemented (variety criterion). This allowed 
gathering information on a broad, though not exhaustive, sample of projects in the sector of interest. 


The questionnaire was developed to detect the relevant aspects of the projects following the Logical 


governance 
BEST PRACTICE 
Nome Progetto Afff 
Nome Progetto Fdd? monitoraggio parteciparione 
— proposta di progetto in corso 
di valutazione 
operatività comunicazione 


Framework Analysis (LFA) scheme. 


Figure 1: Best practices 


The module for the ex-ante multi-criteria evaluation of cultural innovation projects is aimed at de- 
veloping a model of socio-economic analysis of the projects and their potential short and medium-term 
effects, based on multicriteria analysis techniques implemented through analysis dashboards and dy- 
namic simulation of simple and composite georeferenced indicators. The aim is to evaluate the projects 
ex-ante, assigning them specific priorities, in terms of expected socio-economic impact, in order to be 
able to better allocate the available resources (both public/government and crowdfunding). Also this 
module is implemented by properly integrating the Asset software tool into the evaluation workflow 
provided by te HERIBITS platform. 


IMPLEMENTATION 


The techno-social platform is implemented in the form of a web application, adopting a three-tier 
architecture. We used the widely adopted XAMP platform (OSX, Apache, MySQL and PHP) which is 
a valid development environment for web server applications, based on Apple's operating system, the 
open source Apache web server, with license also for commercial use, the very popular open source 
relational database MySQL and the server-side scripting language PHP, in PHP5 version with JSON 
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support, which will be used to implement the business logic. The Symfony web development tools will 
also be used to make business logic software more efficient. 

All the development frameworks adopted are cross-platform and implement the Model View 
Controller (MVC) paradigm, a design pattern that ensures portability and interoperability. We also 
used front-end (ionic2, Angular2JS, ReactJS, vue.js) and back-end frameworks (react2, php + laravel 5, 
nodejs, sails.js). 


i Change 
Notification 


State 


View Selection 


EB EN EN EN 
User Gestures 


=> Method Invocations 
=] => Events 


Figure 2: MVC design pattern 


For the automatic analysis of non quantitative data from the HERIBITS project database, as well as 
to mine unstructured information scraped from external Social Network (SN), we used MongoDB as 
content repository, for its speed in data storage and support for JSON format, a Javascript-based open 
standard format! used to exchange data among distributed web applications and services, widely used 
by SN APIs and SN aggregators. 

MongoDB is a document-oriented NoSQL-Database. MongoDB stores records not in tables as a 
relational database but in BSON documents, which is a binary version of JSON and very similar to the 
object structure. The usage of MongoDB makes his development easier and deployment faster. 


1 JSON (Java Script Object Notation) uses human-readable text to transmit data objects consisting of attribute 
value pairs. It is used exchange to transmit data between a server and web application, as an alternative to XML 
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Figure 3: server and client frameworks with MVC 


In addition, we used ElasticSearch to query MongoDB, since it is very efficient to query large and 
complex databases, again with JSON support. 

Elasticsearch is used to process the vast amount of content in social media. Many recommendation 
engines use Apache Mahout’, we preferred ElasticSearch for a more general support to query data, 
while Mahout is specialized on user profile data (e.g. Mahout is capable of recommending articles 
based on user's preference). In addition, ElasticSearch is dedicated and optimized for data query, and it 
is a perfect complement for MongoDB. 

Elasticsearch is a Java-based search server based on Lucene. It provides a distributed, multitenant- 
capable full-text search engine with a RESTful web interface and schema-free JSON documents. It 
provides scalable search and near real-time search. 

Elasticsearch supports data-intensive distributed applications and implements a computational 
paradigm named MapReduce (via the plugin Taste from Grouplens) The Taste plugin for 
ElasticSearch is Mahout Taste-based Collaborative Filtering implementation, providing the following 
features: (1) Data management for Users/Items/Preferences, (ii) Item-based Recommender, (iii) User- 
based Recommender, (iv) Similar Users/Contents and (v) Text Analysis. 


? Apache Mahout is aimed to produce implementations of distributed machine learning algorithms focused pri- 
marily in the areas of collaborative filtering, clustering and classification. Many of the implementations use the 
Apache Hadoop platform 
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CONCLUSIONS AND FURTHER WORK 


This paper presents the first results of the Heribits project, a research initiative to leverage collective 
awareness and openness in the planning, sourcing and execution of Cultural Heritage initiatives, 
focusing on the technological aspects of the research. 

The project proposes an innovative methodology as well as an advanced technology platform 
enabling a new paradigm for bottom-up and top-down management of cultural heritage initiatives. 

The techno-social platform integrates collaborative tools for CrowdSourcing, as well as analysis 
tools for rating project ideas and evaluate socio-economic impact, to propose best practices and to 
detect similar initiatives in order to avoid project duplications. The platform provides also social 
network capabilities and integrates an ad-hoc CrowdFunding shop. 

The first part of the research was dedicated to re-engineering of the process of cultural project 
proposal and evaluation, through critical examination of real world cases, and with the support of 
domain experts. 

Subsequently, we translated the set of requirements and functional needs emerging from the 
reengineering process into a complete system architecture, defining the technical modules 
implementing the different macro functions. 

Finally we implemented the collaborative techno-social platform, in the form of a web application 
with cooperative nodes, inspired by platforms of collective awareness. The system is currently being 
tested with alfa and beta users, and will be subsequently validated with real users several locations in 
Italy. 
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